Jul 24 17:38:13.163647 ip-10-0-136-89 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 17:38:13.163660 ip-10-0-136-89 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 17:38:13.163669 ip-10-0-136-89 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 17:38:13.163984 ip-10-0-136-89 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 17:38:23.412742 ip-10-0-136-89 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 17:38:23.412760 ip-10-0-136-89 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot fb2301bd861440cca6380be19726deb6 --
Jul 24 17:39:52.673168 ip-10-0-136-89 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 17:39:53.093942 ip-10-0-136-89 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:39:53.093942 ip-10-0-136-89 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 17:39:53.093942 ip-10-0-136-89 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:39:53.093942 ip-10-0-136-89 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 17:39:53.093942 ip-10-0-136-89 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:39:53.095569 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.095483    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 17:39:53.103984 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.103964    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:39:53.103984 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.103981    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:39:53.103984 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.103986    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:39:53.103984 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.103990    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.103994    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.103998    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104002    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104008    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104012    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104016    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104020    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104023    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104027    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104040    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104045    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104048    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104053    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104056    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104060    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104064    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104068    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104072    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104076    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104080    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:39:53.104223 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104084    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104090    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104095    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104100    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104105    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104109    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104113    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104118    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104124    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104130    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104135    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104140    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104144    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104150    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104155    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104159    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104163    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104167    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104172    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:39:53.105107 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104176    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104189    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104193    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104199    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104203    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104208    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104212    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104217    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104221    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104225    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104229    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104234    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104238    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104242    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104246    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104251    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104255    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104259    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104264    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:39:53.105910 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104267    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104272    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104277    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104283    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104289    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104293    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104297    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104304    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104309    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104313    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104318    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104322    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104326    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104330    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104335    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104341    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104346    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104352    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104357    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104361    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:39:53.106694 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104365    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104370    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104374    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104378    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104383    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104387    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104391    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104395    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104400    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104404    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104408    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104412    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104417    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104421    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104426    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104430    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.104434    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105584    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105594    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105600    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:39:53.107563 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105604    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105608    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105613    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105617    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105622    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105626    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105631    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105635    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105639    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105644    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105649    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105654    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105658    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105662    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105666    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105671    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105675    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105679    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105684    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:39:53.108438 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105688    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105693    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105697    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105702    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105708    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105714    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105718    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105722    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105726    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105731    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105735    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105739    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105744    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105748    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105753    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105759    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105765    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105770    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105775    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105779    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:39:53.109101 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105783    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105788    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105792    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105796    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105804    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105809    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105813    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105819    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105824    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105828    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105832    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105836    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105840    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105844    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105848    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105852    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105856    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105862    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105866    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105870    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:39:53.109792 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105901    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105905    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105910    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105914    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105918    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105922    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105926    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105930    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105935    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105939    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105943    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105949    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105955    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105959    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105964    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105969    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105974    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105979    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105983    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:39:53.110317 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105987    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105991    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105995    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.105999    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106004    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106008    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106012    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106017    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106021    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106025    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106030    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106034    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106038    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106043    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106047    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106051    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106055    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106060    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106616    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106623    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:39:53.110795 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106628    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106632    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106637    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106641    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106646    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106650    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106655    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106659    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106663    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106667    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106672    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106676    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106681    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106685    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106689    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106693    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106697    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106701    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106705    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106709    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:39:53.111375 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106713    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106717    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106723    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106728    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106733    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106737    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106741    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106745    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106749    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106754    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106758    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106762    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106767    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106771    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106776    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106780    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106784    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106789    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106793    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106797    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:39:53.111889 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106801    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106805    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106809    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106815    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106819    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106825    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106831    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106835    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106839    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106844    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106848    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106854    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106860    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106865    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106870    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106894    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106899    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106903    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106907    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:39:53.112419 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106911    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106916    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106920    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106924    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106929    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106933    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106937    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106941    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106945    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106950    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106954    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106959    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106962    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106967    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106971    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106976    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106981    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106986    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106991    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:39:53.112907 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.106997    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107001    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107006    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107011    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107015    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107022    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107028    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107033    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107038    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107042    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107047    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107051    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107056    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107060    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107064    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107069    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107073    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107077    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.107081    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107171    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 17:39:53.113371 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107196    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107207    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107214    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107221    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107226    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107232    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107238    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107243    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107250    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107255    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107261    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107267    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107273    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107278    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107283    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107287    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107292    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107299    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107303    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107308    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107313    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107319    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107325    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 17:39:53.113984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107330    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107335    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107340    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107345    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107350    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107355    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107360    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107365    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107371    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107376    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107381    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107386    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107391    2562 flags.go:64] FLAG: --enable-server="true"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107396    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107403    2562 flags.go:64] FLAG: --event-burst="100"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107408    2562 flags.go:64] FLAG: --event-qps="50"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107413    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107418    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107423    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107429    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107433    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107441    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107446    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107451    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107456    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 17:39:53.114539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107461    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107465    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107470    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107475    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107479    2562 flags.go:64] FLAG: --feature-gates=""
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107485    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107490    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107495    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107500    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107505    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107510    2562 flags.go:64] FLAG: --help="false"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107514    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107521    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107526    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107531    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107536    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107542    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107547    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107551    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107556    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107560    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107565    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107570    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107575    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 24 17:39:53.115392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107579    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107583    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107588    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107593    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107599    2562 flags.go:64] FLAG: --lock-file=""
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107605    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107611    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107615    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107624    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107629    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107634    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107639    2562 flags.go:64] FLAG: --logging-format="text"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107643    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107649    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107654    2562 flags.go:64] FLAG: --manifest-url=""
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107658    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107665    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107670    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107676    2562 flags.go:64] FLAG: --max-pods="110"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107681    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107688    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107692    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107697    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107702    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107707    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 17:39:53.116109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107712    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107723    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107728    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107733    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107738    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107743    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107752    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107756    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107761    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107766    2562 flags.go:64] FLAG: --port="10250"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107771    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107775    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a53ac152b0d01aaf"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107782    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107787    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107792    2562 flags.go:64] FLAG: --register-node="true"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107797    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107803    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107808    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107813    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107817    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107823    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107828    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107833    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107838    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107842    2562 flags.go:64] FLAG: --runonce="false"
Jul 24 17:39:53.116725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107847    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107853    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107860    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107865    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107870    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107890    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107895    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107900    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107905    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107910    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107916    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107921    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107926    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107931    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107935    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107944    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107948    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107953    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107959    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107966    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107970    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107976    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107981    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107986    2562 flags.go:64] FLAG: --v="2"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107992    2562 flags.go:64] FLAG: --version="false"
Jul 24 17:39:53.117344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.107999    2562 flags.go:64] FLAG: --vmodule=""
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.108004    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.108010    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108191    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108198    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108203    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108208    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108213    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108218    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108227    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108231    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108236    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108240    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108245    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108249    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108254    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108258    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108262    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108266    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108270    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:39:53.118003 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108275    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108279    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108283    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108312    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108318    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108323    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108330    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108334    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108341    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108347    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108352    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108356    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108361    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108365    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108369    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108375    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108381    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108385    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108390    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:39:53.118484 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108394    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108398    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108404    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108408    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108412    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108417    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108421    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108425    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108430    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108434    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108438    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108442    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108445    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108449    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108453    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108458    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108462    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108466    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108470    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108476    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:39:53.118978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108481    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108485    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108489    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108494    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108499    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108503    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108507    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108512    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108516    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108520    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108524    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108528    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108532    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108537    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108542    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108547    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108551    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108555    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108559    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108563    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:39:53.119466 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108568    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108572    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108576    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108580    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108584    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108588    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108592    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108597    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108602    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108606    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108610    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108616    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108620    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108625    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108629    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108633    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108638    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108642    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108647    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:39:53.119975 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108653    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108659    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108663    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.108667    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.108679    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.117791    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.117806    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.117818    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.117824    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117896    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117902    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117905    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117908    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117911    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117914    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:39:53.120434 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117917    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117920    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117922    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117925    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117928    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117931    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117934    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117936    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117939    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117942    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117944    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117947    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117949    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117952    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117954    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117958    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117960    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117963    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117965    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117968    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:39:53.120805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117971    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117973    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117976    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117979    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117982    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117985    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117988    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117990    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117993    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117995    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.117998    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118000    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118003    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118006    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118010    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118014    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118018    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118022    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118026    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:39:53.121347 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118029    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118035    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118038    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118041    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118044    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118046    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118049    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118052    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118054    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118057    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118060    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118062    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118065    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118067    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118070    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118072    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118075    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118078    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118081    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118084    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:39:53.121805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118086    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118088    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118091    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118094    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118096    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118099    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118101    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118104    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118106    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118109    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118112    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118115    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118117    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118120    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118123    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118126    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118128    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118131    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118133    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118136    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:39:53.122310 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118138    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118141    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118143    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118146    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118148    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118151    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118153    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118156    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118159    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118163    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118166    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118169    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118172    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118175    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.118180    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:39:53.122793 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118291    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118296    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118299    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118302    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118305    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118307    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118310    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118312    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118315    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118317    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118320    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118322    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118326    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118329    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118332    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118334    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118337    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118339    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118342    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118344    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:39:53.123177 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118347    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118350    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118352    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118355    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118358    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118361    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118364    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118366    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118368    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118371    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118374    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118376    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118378    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118381    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118383    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118386    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118388    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118391    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118393    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118396    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:39:53.123654 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118398    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118401    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118403    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118405    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118408    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118411    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118414    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118416    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118419    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118421    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118424    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118426    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118429    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118431    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118434    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118438    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118441    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118444    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118447    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118450    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:39:53.124153 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118453    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118456    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118458    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118461    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118465    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118469    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118473    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118476    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118496    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118500    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118503    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118506    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118509    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118512    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118515    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118518    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118521    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118523    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118527    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:39:53.124631 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118529    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118532    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118542    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118546    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118549    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118552    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118554    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118557    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118560    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118563    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118566    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118568    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118571    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118574    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118576    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118579    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118582    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118584    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118587    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:39:53.125163 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:53.118590    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:39:53.125613 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.118594    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:39:53.125613 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.118689    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 17:39:53.125613 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.122323    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 17:39:53.125613 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.124515    2562 server.go:1019] "Starting client certificate rotation"
Jul 24 17:39:53.125613 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.124618    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:39:53.125613 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.124648    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:39:53.128044 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.128031    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:39:53.129137 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.129124    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:39:53.139510 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.139485    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 24 17:39:53.143834 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.143821    2562 log.go:25] "Validated CRI v1 image API"
Jul 24 17:39:53.144629 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.144611    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 17:39:53.147061 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.147043    2562 fs.go:135] Filesystem UUIDs: map[03d282cb-650b-4a13-9072-26231a4b5d0b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a62b0a46-4b89-4ae2-9b1b-a81dc4ad5172:/dev/nvme0n1p4]
Jul 24 17:39:53.147118 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.147062    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 17:39:53.153485 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.153372    2562 manager.go:216] Machine: {Timestamp:2026-07-24 17:39:53.15209495 +0000 UTC m=+0.356786741 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100467 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23c6467bea120b5bd6de813cc4f1c2 SystemUUID:ec23c646-7bea-120b-5bd6-de813cc4f1c2 BootID:fb2301bd-8614-40cc-a638-0be19726deb6 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:25:f2:b6:5f:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:25:f2:b6:5f:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:b7:69:00:9e:68 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 17:39:53.153485 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.153472    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 17:39:53.153651 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.153568    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 17:39:53.157346 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.157319    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 17:39:53.157430 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.157412    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:39:53.157530 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.157349    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-89.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 17:39:53.158120 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.158107    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 17:39:53.158174 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.158126    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 17:39:53.158174 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.158136    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:39:53.159152 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.159140    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:39:53.159995 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.159984    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 17:39:53.161849 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.161836    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:39:53.161999 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.161988    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 17:39:53.162056 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.162024    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 17:39:53.162056 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.162038    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 17:39:53.162151 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.162058    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 17:39:53.162151 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.162071    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 24 17:39:53.162151 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.162083    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 17:39:53.162473 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.162458    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:39:53.162541 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.162482    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:39:53.165081 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.165065    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 17:39:53.167235 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.167221    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 17:39:53.167293 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.167250    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 17:39:53.169452 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169439    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 17:39:53.169506 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169464    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 17:39:53.169506 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169475    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 17:39:53.169506 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169485    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 17:39:53.169506 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169498    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 17:39:53.169506 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169505    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 17:39:53.169640 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169514    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 17:39:53.169640 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169519    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 17:39:53.169640 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169525    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 17:39:53.169640 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169531    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 17:39:53.169640 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169539    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 17:39:53.169640 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.169544    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 17:39:53.171120 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.171107    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 17:39:53.171120 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.171118    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 17:39:53.173133 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.173115    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-89.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 17:39:53.173211 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.173151    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 17:39:53.173266 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.173238    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-89.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 17:39:53.174121 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.174108    2562 server.go:1268] "Started kubelet"
Jul 24 17:39:53.174744 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.174299    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 17:39:53.174744 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.174349    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 17:39:53.174744 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.174544    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 17:39:53.174904 ip-10-0-136-89 systemd[1]: Started Kubernetes Kubelet.
Jul 24 17:39:53.176281 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.176265    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 17:39:53.177572 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.176705    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 17:39:53.180425 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.180404    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 17:39:53.182973 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.182955    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 17:39:53.183065 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.182971    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 17:39:53.183119 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.183076    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:53.183707 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183688    2562 factory.go:55] Registering systemd factory
Jul 24 17:39:53.183707 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183702    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 17:39:53.183860 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183709    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 24 17:39:53.183860 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183704    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 17:39:53.183860 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183764    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 17:39:53.183860 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183848    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 17:39:53.183860 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183855    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 17:39:53.184093 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183931    2562 factory.go:153] Registering CRI-O factory
Jul 24 17:39:53.184093 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183942    2562 factory.go:223] Registration of the crio container factory successfully
Jul 24 17:39:53.184093 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.183984    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 17:39:53.184093 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.184005    2562 factory.go:103] Registering Raw factory
Jul 24 17:39:53.184093 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.184014    2562 manager.go:1196] Started watching for new ooms in manager
Jul 24 17:39:53.184411 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.184400    2562 manager.go:349] Starting recovery of all containers
Jul 24 17:39:53.189814 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.189776    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-89.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 17:39:53.190058 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.189946    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 17:39:53.191649 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.188079    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-89.ec2.internal.18c54a8c297b1a36  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-89.ec2.internal,UID:ip-10-0-136-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-89.ec2.internal,},FirstTimestamp:2026-07-24 17:39:53.174080054 +0000 UTC m=+0.378771842,LastTimestamp:2026-07-24 17:39:53.174080054 +0000 UTC m=+0.378771842,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-89.ec2.internal,}"
Jul 24 17:39:53.196728 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.196701    2562 manager.go:354] Recovery completed
Jul 24 17:39:53.201921 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.201905    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:39:53.203688 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.203669    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:39:53.203765 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.203703    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:39:53.203765 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.203748    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:39:53.203937 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.203918    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nn2pg"
Jul 24 17:39:53.204242 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.204230    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 17:39:53.204326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.204240    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:39:53.204326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.204242    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 17:39:53.204326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.204279    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:39:53.204916 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.204899    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:39:53.205006 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.204926    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:39:53.205006 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.204942    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:39:53.205703 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.205554    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-89.ec2.internal.18c54a8c2b3ee44d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-89.ec2.internal,UID:ip-10-0-136-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-89.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-89.ec2.internal,},FirstTimestamp:2026-07-24 17:39:53.203688525 +0000 UTC m=+0.408380314,LastTimestamp:2026-07-24 17:39:53.203688525 +0000 UTC m=+0.408380314,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-89.ec2.internal,}"
Jul 24 17:39:53.213110 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.213094    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nn2pg"
Jul 24 17:39:53.214706 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.214645    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-89.ec2.internal.18c54a8c2b3f37fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-89.ec2.internal,UID:ip-10-0-136-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-89.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-89.ec2.internal,},FirstTimestamp:2026-07-24 17:39:53.203709947 +0000 UTC m=+0.408401738,LastTimestamp:2026-07-24 17:39:53.203709947 +0000 UTC m=+0.408401738,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-89.ec2.internal,}"
Jul 24 17:39:53.224376 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.224317    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-89.ec2.internal.18c54a8c2b3fe774  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-89.ec2.internal,UID:ip-10-0-136-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-89.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-89.ec2.internal,},FirstTimestamp:2026-07-24 17:39:53.203754868 +0000 UTC m=+0.408446657,LastTimestamp:2026-07-24 17:39:53.203754868 +0000 UTC m=+0.408446657,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-89.ec2.internal,}"
Jul 24 17:39:53.236162 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.236091    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-89.ec2.internal.18c54a8c2b3ee44d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-89.ec2.internal.18c54a8c2b3ee44d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-89.ec2.internal,UID:ip-10-0-136-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-89.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-89.ec2.internal,},FirstTimestamp:2026-07-24 17:39:53.203688525 +0000 UTC m=+0.408380314,LastTimestamp:2026-07-24 17:39:53.204912707 +0000 UTC m=+0.409604501,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-89.ec2.internal,}"
Jul 24 17:39:53.245790 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.245724    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-89.ec2.internal.18c54a8c2b3f37fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-89.ec2.internal.18c54a8c2b3f37fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-89.ec2.internal,UID:ip-10-0-136-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-89.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-89.ec2.internal,},FirstTimestamp:2026-07-24 17:39:53.203709947 +0000 UTC m=+0.408401738,LastTimestamp:2026-07-24 17:39:53.204932824 +0000 UTC m=+0.409624613,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-89.ec2.internal,}"
Jul 24 17:39:53.255602 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.255531    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-89.ec2.internal.18c54a8c2b3fe774\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-89.ec2.internal.18c54a8c2b3fe774  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-89.ec2.internal,UID:ip-10-0-136-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-89.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-89.ec2.internal,},FirstTimestamp:2026-07-24 17:39:53.203754868 +0000 UTC m=+0.408446657,LastTimestamp:2026-07-24 17:39:53.204947647 +0000 UTC m=+0.409639435,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-89.ec2.internal,}"
Jul 24 17:39:53.283662 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.283624    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:53.349252 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.349189    2562 policy_none.go:49] "None policy: Start"
Jul 24 17:39:53.349252 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.349216    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 17:39:53.349252 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.349229    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 17:39:53.350385 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.350372    2562 policy_none.go:47] "Start"
Jul 24 17:39:53.378622 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.378440    2562 manager.go:337] "Starting Device Plugin manager"
Jul 24 17:39:53.378719 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.378645    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 17:39:53.378719 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.378658    2562 server.go:87] "Starting device plugin registration server"
Jul 24 17:39:53.378920 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.378909    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 17:39:53.378958 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.378923    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 17:39:53.379022 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.379008    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 17:39:53.379096 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.379086    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 17:39:53.379146 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.379098    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 17:39:53.379632 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.379613    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 17:39:53.379709 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.379657    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:53.394997 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.394977    2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-89.ec2.internal\" not found" node="ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.399575 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.399555    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 17:39:53.400731 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.400715    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 17:39:53.400795 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.400746    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 17:39:53.400795 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.400763    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 17:39:53.400795 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.400791    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 17:39:53.402984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.402967    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:39:53.479371 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.479346    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:39:53.480230 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.480212    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:39:53.480298 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.480242    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:39:53.480298 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.480254    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:39:53.480298 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.480277    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.489132 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.489106    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.489132 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.489136    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-89.ec2.internal\": node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:53.501150 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.501127    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"]
Jul 24 17:39:53.501232 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.501179    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:39:53.502604 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.502590    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:39:53.502676 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.502616    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:39:53.502676 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.502626    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:39:53.504586 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.504574    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:39:53.504685 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.504672    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.504720 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.504713    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:39:53.505226 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.505214    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:39:53.505282 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.505237    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:39:53.505282 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.505253    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:39:53.506057 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.506042    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:39:53.506125 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.506071    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:39:53.506125 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.506084    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:39:53.508138 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.508123    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.508214 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.508154    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:39:53.509663 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.509648    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:39:53.509721 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.509674    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:39:53.509721 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.509691    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:39:53.513136 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.513123    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:53.525416 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.525399    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-89.ec2.internal\" not found" node="ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.529980 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.529957    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-89.ec2.internal\" not found" node="ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.586227 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.586202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd13083ddd3ee48db23d4b5c37fef9a4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-89.ec2.internal\" (UID: \"dd13083ddd3ee48db23d4b5c37fef9a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.586313 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.586229    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e6877ce27ede56185f497d7314010a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal\" (UID: \"3e6877ce27ede56185f497d7314010a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.586313 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.586246    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e6877ce27ede56185f497d7314010a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal\" (UID: \"3e6877ce27ede56185f497d7314010a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.614170 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.614130    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:53.686803 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.686780    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd13083ddd3ee48db23d4b5c37fef9a4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-89.ec2.internal\" (UID: \"dd13083ddd3ee48db23d4b5c37fef9a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.686857 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.686824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd13083ddd3ee48db23d4b5c37fef9a4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-89.ec2.internal\" (UID: \"dd13083ddd3ee48db23d4b5c37fef9a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.686857 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.686850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e6877ce27ede56185f497d7314010a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal\" (UID: \"3e6877ce27ede56185f497d7314010a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.686941 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.686868    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e6877ce27ede56185f497d7314010a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal\" (UID: \"3e6877ce27ede56185f497d7314010a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.686941 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.686897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e6877ce27ede56185f497d7314010a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal\" (UID: \"3e6877ce27ede56185f497d7314010a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.687000 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.686940    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e6877ce27ede56185f497d7314010a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal\" (UID: \"3e6877ce27ede56185f497d7314010a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.714923 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.714898    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:53.815378 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.815356    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:53.828532 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.828518    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.833624 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:53.833183    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:53.916356 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:53.916309    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:54.016771 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.016734    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:54.116851 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.116822    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-89.ec2.internal\" not found"
Jul 24 17:39:54.117811 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.117778    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:39:54.123218 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.123193    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 17:39:54.123356 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.123336    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 17:39:54.123403 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.123379    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 17:39:54.154386 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.154362    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:39:54.162728 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.162708    2562 apiserver.go:52] "Watching apiserver"
Jul 24 17:39:54.170818 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.170759    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 17:39:54.171149 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.171126    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-mf7bb","openshift-network-operator/iptables-alerter-nlgzt","openshift-ovn-kubernetes/ovnkube-node-r59tv","kube-system/global-pull-secret-syncer-8fsnd","kube-system/konnectivity-agent-p9kck","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq","openshift-cluster-node-tuning-operator/tuned-95ljs","openshift-dns/node-resolver-t4rbd","openshift-multus/multus-additional-cni-plugins-tzghr","openshift-image-registry/node-ca-t4t99","openshift-multus/multus-pl7lv","openshift-multus/network-metrics-daemon-kz48t"]
Jul 24 17:39:54.173738 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.173721    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.175973 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.175937    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:39:54.176215 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.176188    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 17:39:54.176460 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.176442    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 17:39:54.176763 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.176746    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wb2tl\""
Jul 24 17:39:54.176763 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.176757    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 17:39:54.176951 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.176774    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 17:39:54.176951 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.176829    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 17:39:54.176951 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.176835    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 17:39:54.178050 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.178033    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 17:39:54.178296 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.178276    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.178385 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.178304    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 17:39:54.178679 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.178663    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8dt9x\""
Jul 24 17:39:54.180471 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.180450    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-snznd\""
Jul 24 17:39:54.180471 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.180465    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 17:39:54.180602 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.180454    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 17:39:54.180730 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.180715    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.182657 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.182634    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rcrbd\""
Jul 24 17:39:54.182763 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.182634    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 17:39:54.182763 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.182640    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 17:39:54.182858 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.182798    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 17:39:54.183055 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.183037    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 17:39:54.183106 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.183071    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:39:54.183160 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.183137    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:39:54.183220 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.183158    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:54.186683 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.186665    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.189183 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189151    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:39:54.189585 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189565    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.189731 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-run-systemd\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.189796 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189751    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-node-log\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.189796 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189787    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81d447a3-e3df-4850-bcd4-60ef6e80cbcd-host\") pod \"node-ca-t4t99\" (UID: \"81d447a3-e3df-4850-bcd4-60ef6e80cbcd\") " pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.189899 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189817    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-cni-bin\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.189899 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjbdd\" (UniqueName: \"kubernetes.io/projected/edf57894-9068-4294-952a-76003b90530f-kube-api-access-jjbdd\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.189899 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189882    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsvxc\" (UniqueName: \"kubernetes.io/projected/291c612c-f9bd-4525-a4b6-e3f6777def41-kube-api-access-gsvxc\") pod \"node-resolver-t4rbd\" (UID: \"291c612c-f9bd-4525-a4b6-e3f6777def41\") " pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.190081 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-cni-netd\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190081 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189975    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/edf57894-9068-4294-952a-76003b90530f-env-overrides\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190081 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190009    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aec7a6de-7be2-41cc-b3f1-d447f0bbdab3-konnectivity-ca\") pod \"konnectivity-agent-p9kck\" (UID: \"aec7a6de-7be2-41cc-b3f1-d447f0bbdab3\") " pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:39:54.190081 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190050    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/81d447a3-e3df-4850-bcd4-60ef6e80cbcd-serviceca\") pod \"node-ca-t4t99\" (UID: \"81d447a3-e3df-4850-bcd4-60ef6e80cbcd\") " pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.190286 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190084    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-systemd-units\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190286 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190111    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-run-openvswitch\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190286 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190141    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aec7a6de-7be2-41cc-b3f1-d447f0bbdab3-agent-certs\") pod \"konnectivity-agent-p9kck\" (UID: \"aec7a6de-7be2-41cc-b3f1-d447f0bbdab3\") " pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:39:54.190286 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190171    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/291c612c-f9bd-4525-a4b6-e3f6777def41-tmp-dir\") pod \"node-resolver-t4rbd\" (UID: \"291c612c-f9bd-4525-a4b6-e3f6777def41\") " pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.190286 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190205    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-var-lib-openvswitch\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190286 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190237    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-run-ovn-kubernetes\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190286 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190262    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/edf57894-9068-4294-952a-76003b90530f-ovn-node-metrics-cert\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190296    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/291c612c-f9bd-4525-a4b6-e3f6777def41-hosts-file\") pod \"node-resolver-t4rbd\" (UID: \"291c612c-f9bd-4525-a4b6-e3f6777def41\") " pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190330    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-slash\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-run-ovn\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-log-socket\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28f75bbf-9ea5-4984-ba60-2f464cda35fd-host-slash\") pod \"iptables-alerter-nlgzt\" (UID: \"28f75bbf-9ea5-4984-ba60-2f464cda35fd\") " pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190442    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpg22\" (UniqueName: \"kubernetes.io/projected/28f75bbf-9ea5-4984-ba60-2f464cda35fd-kube-api-access-kpg22\") pod \"iptables-alerter-nlgzt\" (UID: \"28f75bbf-9ea5-4984-ba60-2f464cda35fd\") " pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190473    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-kubelet\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-run-netns\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.190594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190564    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/edf57894-9068-4294-952a-76003b90530f-ovnkube-config\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.191083 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189584    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qfgz4\""
Jul 24 17:39:54.191083 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190597    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkcd4\" (UniqueName: \"kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4\") pod \"network-check-target-mf7bb\" (UID: \"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b\") " pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:39:54.191083 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190829    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28f75bbf-9ea5-4984-ba60-2f464cda35fd-iptables-alerter-script\") pod \"iptables-alerter-nlgzt\" (UID: \"28f75bbf-9ea5-4984-ba60-2f464cda35fd\") " pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.191083 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189646    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 17:39:54.191083 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-etc-openvswitch\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.191083 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.190994    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/edf57894-9068-4294-952a-76003b90530f-ovnkube-script-lib\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.191083 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.191053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzz7l\" (UniqueName: \"kubernetes.io/projected/81d447a3-e3df-4850-bcd4-60ef6e80cbcd-kube-api-access-tzz7l\") pod \"node-ca-t4t99\" (UID: \"81d447a3-e3df-4850-bcd4-60ef6e80cbcd\") " pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.191421 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.189584    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 17:39:54.191421 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.191398    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:39:54.192367 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.192350    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.194526 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.194509    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 17:39:54.194641 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.194620    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 17:39:54.194699 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.194620    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ksktd\""
Jul 24 17:39:54.194699 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.194626    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 17:39:54.194867 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.194853    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.195252 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.195238    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:39:54.196950 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.196936    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:39:54.197082 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.197067    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d8tkk\""
Jul 24 17:39:54.197448 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.197435    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 17:39:54.197628 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.197612    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.200034 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.200016    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.200117 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.200103    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:39:54.200211 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.200195    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal"
Jul 24 17:39:54.200272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.200216    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 17:39:54.200525 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.200504    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 17:39:54.200620 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.200573    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 17:39:54.200674 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.200658    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ttg2z\""
Jul 24 17:39:54.200768 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.200753    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 17:39:54.200841 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.200828    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 17:39:54.202037 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.202023    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 17:39:54.202465 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.202450    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal"]
Jul 24 17:39:54.202553 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.202541    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:54.202615 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.202585    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:39:54.202615 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.202590    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q658w\""
Jul 24 17:39:54.210771 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.210722    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:39:54.210899 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.210874    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal"]
Jul 24 17:39:54.226592 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.226572    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6vtd"
Jul 24 17:39:54.234721 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.234706    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6vtd"
Jul 24 17:39:54.284858 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.284840    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 17:39:54.291611 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291596    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-lib-modules\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.291668 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291623    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-slash\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.291668 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-run-ovn\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.291668 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpg22\" (UniqueName: \"kubernetes.io/projected/28f75bbf-9ea5-4984-ba60-2f464cda35fd-kube-api-access-kpg22\") pod \"iptables-alerter-nlgzt\" (UID: \"28f75bbf-9ea5-4984-ba60-2f464cda35fd\") " pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.291782 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-run-netns\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.291782 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291711    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-run-netns\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.291782 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-slash\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.291782 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291750    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-run-ovn\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.291782 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291772    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.291988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291790    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-cnibin\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.291988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4707a45-429b-4d07-a7b9-41d3bcf570fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.291988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291829    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-run-k8s-cni-cncf-io\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.291988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-modprobe-d\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.291988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lwgv\" (UniqueName: \"kubernetes.io/projected/64665c61-db20-4439-a490-af90a1d0e579-kube-api-access-4lwgv\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:54.291988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.291939    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-etc-openvswitch\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292255 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292010    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/edf57894-9068-4294-952a-76003b90530f-ovnkube-script-lib\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292255 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzz7l\" (UniqueName: \"kubernetes.io/projected/81d447a3-e3df-4850-bcd4-60ef6e80cbcd-kube-api-access-tzz7l\") pod \"node-ca-t4t99\" (UID: \"81d447a3-e3df-4850-bcd4-60ef6e80cbcd\") " pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.292255 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292076    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-etc-openvswitch\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292255 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/74b9bf45-8bba-459e-bd93-ee92f990ae95-kubelet-config\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.292255 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292184    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-os-release\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.292255 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-run-multus-certs\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.292255 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-etc-kubernetes\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.292540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292300    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-node-log\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-device-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.292540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292387    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-node-log\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292416    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4707a45-429b-4d07-a7b9-41d3bcf570fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.292540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-os-release\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.292540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292459    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-sysctl-conf\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.292540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-tuned\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.292540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292490    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-cni-bin\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292549    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-cni-bin\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292604    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjbdd\" (UniqueName: \"kubernetes.io/projected/edf57894-9068-4294-952a-76003b90530f-kube-api-access-jjbdd\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxzpt\" (UniqueName: \"kubernetes.io/projected/c4707a45-429b-4d07-a7b9-41d3bcf570fa-kube-api-access-sxzpt\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-cni-netd\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/edf57894-9068-4294-952a-76003b90530f-env-overrides\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292722    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bee228fc-b4ca-41fe-8bf2-f497792f028a-tmp\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292737    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aec7a6de-7be2-41cc-b3f1-d447f0bbdab3-agent-certs\") pod \"konnectivity-agent-p9kck\" (UID: \"aec7a6de-7be2-41cc-b3f1-d447f0bbdab3\") " pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292736    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-cni-netd\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/291c612c-f9bd-4525-a4b6-e3f6777def41-tmp-dir\") pod \"node-resolver-t4rbd\" (UID: \"291c612c-f9bd-4525-a4b6-e3f6777def41\") " pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292796    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-system-cni-dir\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292817    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-daemon-config\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292864    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-var-lib-openvswitch\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292878    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/edf57894-9068-4294-952a-76003b90530f-ovnkube-script-lib\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292936    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-run-ovn-kubernetes\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-var-lib-openvswitch\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.292985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.292970    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/edf57894-9068-4294-952a-76003b90530f-ovn-node-metrics-cert\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293079    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/291c612c-f9bd-4525-a4b6-e3f6777def41-tmp-dir\") pod \"node-resolver-t4rbd\" (UID: \"291c612c-f9bd-4525-a4b6-e3f6777def41\") " pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293134    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/edf57894-9068-4294-952a-76003b90530f-env-overrides\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/291c612c-f9bd-4525-a4b6-e3f6777def41-hosts-file\") pod \"node-resolver-t4rbd\" (UID: \"291c612c-f9bd-4525-a4b6-e3f6777def41\") " pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293246    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-run-ovn-kubernetes\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293241    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-socket-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293315    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-system-cni-dir\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293361    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/291c612c-f9bd-4525-a4b6-e3f6777def41-hosts-file\") pod \"node-resolver-t4rbd\" (UID: \"291c612c-f9bd-4525-a4b6-e3f6777def41\") " pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-var-lib-cni-bin\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-sysconfig\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-log-socket\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293460    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28f75bbf-9ea5-4984-ba60-2f464cda35fd-host-slash\") pod \"iptables-alerter-nlgzt\" (UID: \"28f75bbf-9ea5-4984-ba60-2f464cda35fd\") " pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293492    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-log-socket\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/74b9bf45-8bba-459e-bd93-ee92f990ae95-dbus\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293510    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28f75bbf-9ea5-4984-ba60-2f464cda35fd-host-slash\") pod \"iptables-alerter-nlgzt\" (UID: \"28f75bbf-9ea5-4984-ba60-2f464cda35fd\") " pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293542    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.293557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293561    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-cni-dir\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.294292 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293598    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-kubelet\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.294292 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293636    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-kubelet\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.294292 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.294292 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/edf57894-9068-4294-952a-76003b90530f-ovnkube-config\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.294292 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293755    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.294292 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.293790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkcd4\" (UniqueName: \"kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4\") pod \"network-check-target-mf7bb\" (UID: \"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b\") " pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:39:54.294967 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.294935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/edf57894-9068-4294-952a-76003b90530f-ovnkube-config\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.295102 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.295040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28f75bbf-9ea5-4984-ba60-2f464cda35fd-iptables-alerter-script\") pod \"iptables-alerter-nlgzt\" (UID: \"28f75bbf-9ea5-4984-ba60-2f464cda35fd\") " pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.295185 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.295121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5a8021bc-a6d1-4d9b-be64-b04494e23e90-cni-binary-copy\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.295185 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.295162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-var-lib-kubelet\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.295315 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.295220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-conf-dir\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.295728 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.295696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28f75bbf-9ea5-4984-ba60-2f464cda35fd-iptables-alerter-script\") pod \"iptables-alerter-nlgzt\" (UID: \"28f75bbf-9ea5-4984-ba60-2f464cda35fd\") " pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.295886 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.295866    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.296094 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.295934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-hostroot\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.296094 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.295971    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-kubernetes\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.296094 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-systemd\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.296094 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfnrf\" (UniqueName: \"kubernetes.io/projected/bee228fc-b4ca-41fe-8bf2-f497792f028a-kube-api-access-hfnrf\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.296094 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296075    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-run-systemd\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.296413 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81d447a3-e3df-4850-bcd4-60ef6e80cbcd-host\") pod \"node-ca-t4t99\" (UID: \"81d447a3-e3df-4850-bcd4-60ef6e80cbcd\") " pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.296413 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-var-lib-kubelet\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.296413 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsvxc\" (UniqueName: \"kubernetes.io/projected/291c612c-f9bd-4525-a4b6-e3f6777def41-kube-api-access-gsvxc\") pod \"node-resolver-t4rbd\" (UID: \"291c612c-f9bd-4525-a4b6-e3f6777def41\") " pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.296413 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296217    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4707a45-429b-4d07-a7b9-41d3bcf570fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.296413 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296244    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-cnibin\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.296413 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296280    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-host\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.296413 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aec7a6de-7be2-41cc-b3f1-d447f0bbdab3-konnectivity-ca\") pod \"konnectivity-agent-p9kck\" (UID: \"aec7a6de-7be2-41cc-b3f1-d447f0bbdab3\") " pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:39:54.296413 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/81d447a3-e3df-4850-bcd4-60ef6e80cbcd-serviceca\") pod \"node-ca-t4t99\" (UID: \"81d447a3-e3df-4850-bcd4-60ef6e80cbcd\") " pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.296413 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-registration-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-sys-fs\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296475    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5ps6\" (UniqueName: \"kubernetes.io/projected/9248052a-97d7-41c5-9826-8485ec8dd504-kube-api-access-l5ps6\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296507    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-run\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296542    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-systemd-units\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-run-openvswitch\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296598    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-run-netns\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-var-lib-cni-multus\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296665    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-sysctl-d\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296694    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-sys\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296719    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296755    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-socket-dir-parent\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.296829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296791    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwkbf\" (UniqueName: \"kubernetes.io/projected/5a8021bc-a6d1-4d9b-be64-b04494e23e90-kube-api-access-qwkbf\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.297439 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296985    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-run-systemd\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.297439 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.296995    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81d447a3-e3df-4850-bcd4-60ef6e80cbcd-host\") pod \"node-ca-t4t99\" (UID: \"81d447a3-e3df-4850-bcd4-60ef6e80cbcd\") " pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.297439 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.297242    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/edf57894-9068-4294-952a-76003b90530f-ovn-node-metrics-cert\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.297439 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.297382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-systemd-units\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.297641 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.297447    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edf57894-9068-4294-952a-76003b90530f-run-openvswitch\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.298620 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.297925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aec7a6de-7be2-41cc-b3f1-d447f0bbdab3-konnectivity-ca\") pod \"konnectivity-agent-p9kck\" (UID: \"aec7a6de-7be2-41cc-b3f1-d447f0bbdab3\") " pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:39:54.298620 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.298094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aec7a6de-7be2-41cc-b3f1-d447f0bbdab3-agent-certs\") pod \"konnectivity-agent-p9kck\" (UID: \"aec7a6de-7be2-41cc-b3f1-d447f0bbdab3\") " pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:39:54.298620 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.298176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/81d447a3-e3df-4850-bcd4-60ef6e80cbcd-serviceca\") pod \"node-ca-t4t99\" (UID: \"81d447a3-e3df-4850-bcd4-60ef6e80cbcd\") " pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.300224 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.300205    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:39:54.300324 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.300228    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:39:54.300324 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.300239    2562 projected.go:196] Error preparing data for projected volume kube-api-access-fkcd4 for pod openshift-network-diagnostics/network-check-target-mf7bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:39:54.300324 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.300317    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4 podName:0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b nodeName:}" failed. No retries permitted until 2026-07-24 17:39:54.800289671 +0000 UTC m=+2.004981449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fkcd4" (UniqueName: "kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4") pod "network-check-target-mf7bb" (UID: "0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:39:54.301002 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.300982    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpg22\" (UniqueName: \"kubernetes.io/projected/28f75bbf-9ea5-4984-ba60-2f464cda35fd-kube-api-access-kpg22\") pod \"iptables-alerter-nlgzt\" (UID: \"28f75bbf-9ea5-4984-ba60-2f464cda35fd\") " pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.301246 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.301226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzz7l\" (UniqueName: \"kubernetes.io/projected/81d447a3-e3df-4850-bcd4-60ef6e80cbcd-kube-api-access-tzz7l\") pod \"node-ca-t4t99\" (UID: \"81d447a3-e3df-4850-bcd4-60ef6e80cbcd\") " pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.302154 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.302135    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjbdd\" (UniqueName: \"kubernetes.io/projected/edf57894-9068-4294-952a-76003b90530f-kube-api-access-jjbdd\") pod \"ovnkube-node-r59tv\" (UID: \"edf57894-9068-4294-952a-76003b90530f\") " pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.311829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.311809    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsvxc\" (UniqueName: \"kubernetes.io/projected/291c612c-f9bd-4525-a4b6-e3f6777def41-kube-api-access-gsvxc\") pod \"node-resolver-t4rbd\" (UID: \"291c612c-f9bd-4525-a4b6-e3f6777def41\") " pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.350419 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.350357    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 17:34:53 +0000 UTC" deadline="2028-04-08 21:50:26.131172166 +0000 UTC"
Jul 24 17:39:54.350419 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.350413    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14980h10m31.780762378s"
Jul 24 17:39:54.397653 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5a8021bc-a6d1-4d9b-be64-b04494e23e90-cni-binary-copy\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397658    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-var-lib-kubelet\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-conf-dir\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397700    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-hostroot\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-kubernetes\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397761    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-var-lib-kubelet\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-systemd\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfnrf\" (UniqueName: \"kubernetes.io/projected/bee228fc-b4ca-41fe-8bf2-f497792f028a-kube-api-access-hfnrf\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397823    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-conf-dir\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.397854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-var-lib-kubelet\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-hostroot\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397884    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4707a45-429b-4d07-a7b9-41d3bcf570fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397838    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-kubernetes\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397907    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-var-lib-kubelet\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-cnibin\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-systemd\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-host\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397979    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-registration-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398006    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-sys-fs\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398021    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-registration-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397981    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-cnibin\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5ps6\" (UniqueName: \"kubernetes.io/projected/9248052a-97d7-41c5-9826-8485ec8dd504-kube-api-access-l5ps6\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.397983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-host\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398078    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-run\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398095    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-sys-fs\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-run-netns\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398158    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-var-lib-cni-multus\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.398470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398161    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-run\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-sysctl-d\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-var-lib-cni-multus\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398218    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-sys\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-run-netns\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-socket-dir-parent\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-sysctl-d\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5a8021bc-a6d1-4d9b-be64-b04494e23e90-cni-binary-copy\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-sys\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwkbf\" (UniqueName: \"kubernetes.io/projected/5a8021bc-a6d1-4d9b-be64-b04494e23e90-kube-api-access-qwkbf\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-socket-dir-parent\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-lib-modules\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.398377    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-lib-modules\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.398435    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs podName:64665c61-db20-4439-a490-af90a1d0e579 nodeName:}" failed. No retries permitted until 2026-07-24 17:39:54.898415066 +0000 UTC m=+2.103106864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs") pod "network-metrics-daemon-kz48t" (UID: "64665c61-db20-4439-a490-af90a1d0e579") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:39:54.399368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-cnibin\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-cnibin\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4707a45-429b-4d07-a7b9-41d3bcf570fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.398487    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-run-k8s-cni-cncf-io\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398525    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4707a45-429b-4d07-a7b9-41d3bcf570fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.398540    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret podName:74b9bf45-8bba-459e-bd93-ee92f990ae95 nodeName:}" failed. No retries permitted until 2026-07-24 17:39:54.898522231 +0000 UTC m=+2.103214019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret") pod "global-pull-secret-syncer-8fsnd" (UID: "74b9bf45-8bba-459e-bd93-ee92f990ae95") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-run-k8s-cni-cncf-io\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-modprobe-d\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lwgv\" (UniqueName: \"kubernetes.io/projected/64665c61-db20-4439-a490-af90a1d0e579-kube-api-access-4lwgv\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/74b9bf45-8bba-459e-bd93-ee92f990ae95-kubelet-config\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398651    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-os-release\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398675    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-modprobe-d\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-run-multus-certs\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398697    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/74b9bf45-8bba-459e-bd93-ee92f990ae95-kubelet-config\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-etc-kubernetes\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-os-release\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.399986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-device-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-run-multus-certs\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398756    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-etc-kubernetes\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4707a45-429b-4d07-a7b9-41d3bcf570fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-os-release\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-device-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-sysctl-conf\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-tuned\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-os-release\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxzpt\" (UniqueName: \"kubernetes.io/projected/c4707a45-429b-4d07-a7b9-41d3bcf570fa-kube-api-access-sxzpt\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398916    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4707a45-429b-4d07-a7b9-41d3bcf570fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bee228fc-b4ca-41fe-8bf2-f497792f028a-tmp\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-sysctl-conf\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.398978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-system-cni-dir\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399010    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-system-cni-dir\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399025    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-daemon-config\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.400504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399107    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-socket-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-system-cni-dir\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-var-lib-cni-bin\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399185    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-sysconfig\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/74b9bf45-8bba-459e-bd93-ee92f990ae95-dbus\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399243    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4707a45-429b-4d07-a7b9-41d3bcf570fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-cni-dir\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399302    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399344    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-host-var-lib-cni-bin\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4707a45-429b-4d07-a7b9-41d3bcf570fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9248052a-97d7-41c5-9826-8485ec8dd504-socket-dir\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399396    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-sysconfig\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399445    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-system-cni-dir\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399447    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-cni-dir\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/74b9bf45-8bba-459e-bd93-ee92f990ae95-dbus\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.399629    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5a8021bc-a6d1-4d9b-be64-b04494e23e90-multus-daemon-config\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.401089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.401023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bee228fc-b4ca-41fe-8bf2-f497792f028a-tmp\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.401609 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.401145    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bee228fc-b4ca-41fe-8bf2-f497792f028a-etc-tuned\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.406880 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.406858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfnrf\" (UniqueName: \"kubernetes.io/projected/bee228fc-b4ca-41fe-8bf2-f497792f028a-kube-api-access-hfnrf\") pod \"tuned-95ljs\" (UID: \"bee228fc-b4ca-41fe-8bf2-f497792f028a\") " pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.407394 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.407372    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwkbf\" (UniqueName: \"kubernetes.io/projected/5a8021bc-a6d1-4d9b-be64-b04494e23e90-kube-api-access-qwkbf\") pod \"multus-pl7lv\" (UID: \"5a8021bc-a6d1-4d9b-be64-b04494e23e90\") " pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.408770 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.408744    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lwgv\" (UniqueName: \"kubernetes.io/projected/64665c61-db20-4439-a490-af90a1d0e579-kube-api-access-4lwgv\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:54.408857 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.408810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxzpt\" (UniqueName: \"kubernetes.io/projected/c4707a45-429b-4d07-a7b9-41d3bcf570fa-kube-api-access-sxzpt\") pod \"multus-additional-cni-plugins-tzghr\" (UID: \"c4707a45-429b-4d07-a7b9-41d3bcf570fa\") " pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.408857 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.408810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5ps6\" (UniqueName: \"kubernetes.io/projected/9248052a-97d7-41c5-9826-8485ec8dd504-kube-api-access-l5ps6\") pod \"aws-ebs-csi-driver-node-7ncrq\" (UID: \"9248052a-97d7-41c5-9826-8485ec8dd504\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.483284 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.483252    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd13083ddd3ee48db23d4b5c37fef9a4.slice/crio-aceca378fe6a6815b01bbc4e83a036dc61e7599af4c9ccc865bc81d7aa9a41a0 WatchSource:0}: Error finding container aceca378fe6a6815b01bbc4e83a036dc61e7599af4c9ccc865bc81d7aa9a41a0: Status 404 returned error can't find the container with id aceca378fe6a6815b01bbc4e83a036dc61e7599af4c9ccc865bc81d7aa9a41a0
Jul 24 17:39:54.490597 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.490575    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:39:54.493971 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.493929    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:39:54.494254 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.494237    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e6877ce27ede56185f497d7314010a0.slice/crio-0e0945b95ddb55b7a841a6607cc3c5bf2db624329275065f7dd1bd80803f993a WatchSource:0}: Error finding container 0e0945b95ddb55b7a841a6607cc3c5bf2db624329275065f7dd1bd80803f993a: Status 404 returned error can't find the container with id 0e0945b95ddb55b7a841a6607cc3c5bf2db624329275065f7dd1bd80803f993a
Jul 24 17:39:54.502736 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.502718    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedf57894_9068_4294_952a_76003b90530f.slice/crio-0b9c019f18da422c372a29bef5c44ea0a5a47d7a300aa236abe071d447ecdaea WatchSource:0}: Error finding container 0b9c019f18da422c372a29bef5c44ea0a5a47d7a300aa236abe071d447ecdaea: Status 404 returned error can't find the container with id 0b9c019f18da422c372a29bef5c44ea0a5a47d7a300aa236abe071d447ecdaea
Jul 24 17:39:54.510686 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.510669    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:39:54.519083 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.519059    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec7a6de_7be2_41cc_b3f1_d447f0bbdab3.slice/crio-9a57e8acb96d29f5886134cc0d430313738b286a12ff5cedc22d19e6dd4d23d6 WatchSource:0}: Error finding container 9a57e8acb96d29f5886134cc0d430313738b286a12ff5cedc22d19e6dd4d23d6: Status 404 returned error can't find the container with id 9a57e8acb96d29f5886134cc0d430313738b286a12ff5cedc22d19e6dd4d23d6
Jul 24 17:39:54.530410 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.530389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t4t99"
Jul 24 17:39:54.531568 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.531552    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t4rbd"
Jul 24 17:39:54.536727 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.536707    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nlgzt"
Jul 24 17:39:54.541670 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.541650    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d447a3_e3df_4850_bcd4_60ef6e80cbcd.slice/crio-a76ed2ccb16991b0a8db4e36d884854e8e2f09f5b7f02a4a9466a6f90a7971cb WatchSource:0}: Error finding container a76ed2ccb16991b0a8db4e36d884854e8e2f09f5b7f02a4a9466a6f90a7971cb: Status 404 returned error can't find the container with id a76ed2ccb16991b0a8db4e36d884854e8e2f09f5b7f02a4a9466a6f90a7971cb
Jul 24 17:39:54.544700 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.544676    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291c612c_f9bd_4525_a4b6_e3f6777def41.slice/crio-992ec8bb6c73cdfdcea18cce7632e12d705991354bb4a4e7f22d1b6daa9bf451 WatchSource:0}: Error finding container 992ec8bb6c73cdfdcea18cce7632e12d705991354bb4a4e7f22d1b6daa9bf451: Status 404 returned error can't find the container with id 992ec8bb6c73cdfdcea18cce7632e12d705991354bb4a4e7f22d1b6daa9bf451
Jul 24 17:39:54.547229 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.547207    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f75bbf_9ea5_4984_ba60_2f464cda35fd.slice/crio-50859d54dfa0bec05e50f2da843010efcb86e976e50d26b51927edc5de77273d WatchSource:0}: Error finding container 50859d54dfa0bec05e50f2da843010efcb86e976e50d26b51927edc5de77273d: Status 404 returned error can't find the container with id 50859d54dfa0bec05e50f2da843010efcb86e976e50d26b51927edc5de77273d
Jul 24 17:39:54.565283 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.565264    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq"
Jul 24 17:39:54.573978 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.573947    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9248052a_97d7_41c5_9826_8485ec8dd504.slice/crio-10b81309d0fd74550a65ef32477096a8fb829292d79c139468c32bc0e66d4ee9 WatchSource:0}: Error finding container 10b81309d0fd74550a65ef32477096a8fb829292d79c139468c32bc0e66d4ee9: Status 404 returned error can't find the container with id 10b81309d0fd74550a65ef32477096a8fb829292d79c139468c32bc0e66d4ee9
Jul 24 17:39:54.577092 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.577076    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-95ljs"
Jul 24 17:39:54.584753 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.584734    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee228fc_b4ca_41fe_8bf2_f497792f028a.slice/crio-b9cea85d7b88a11719f679e20141dfeb62fc3ea73b1c366d6169a921bf2f3e4b WatchSource:0}: Error finding container b9cea85d7b88a11719f679e20141dfeb62fc3ea73b1c366d6169a921bf2f3e4b: Status 404 returned error can't find the container with id b9cea85d7b88a11719f679e20141dfeb62fc3ea73b1c366d6169a921bf2f3e4b
Jul 24 17:39:54.597719 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.597700    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tzghr"
Jul 24 17:39:54.606055 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.606025    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4707a45_429b_4d07_a7b9_41d3bcf570fa.slice/crio-eb294be7f127c635bb33753626d081de83d51d77c2ad060323545b34ec5dd383 WatchSource:0}: Error finding container eb294be7f127c635bb33753626d081de83d51d77c2ad060323545b34ec5dd383: Status 404 returned error can't find the container with id eb294be7f127c635bb33753626d081de83d51d77c2ad060323545b34ec5dd383
Jul 24 17:39:54.612105 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.612089    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pl7lv"
Jul 24 17:39:54.620777 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:39:54.620757    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8021bc_a6d1_4d9b_be64_b04494e23e90.slice/crio-f2aa3d2e65e455fb21ef0c39f1556430618452fd64065e324bfdd9b00168f94a WatchSource:0}: Error finding container f2aa3d2e65e455fb21ef0c39f1556430618452fd64065e324bfdd9b00168f94a: Status 404 returned error can't find the container with id f2aa3d2e65e455fb21ef0c39f1556430618452fd64065e324bfdd9b00168f94a
Jul 24 17:39:54.622277 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.622260    2562 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-pl7lv_openshift-multus(5a8021bc-a6d1-4d9b-be64-b04494e23e90): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 24 17:39:54.622321 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.622291    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-pl7lv" podUID="5a8021bc-a6d1-4d9b-be64-b04494e23e90"
Jul 24 17:39:54.625375 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.625357    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:39:54.802427 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.802409    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkcd4\" (UniqueName: \"kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4\") pod \"network-check-target-mf7bb\" (UID: \"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b\") " pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:39:54.802520 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.802508    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:39:54.802559 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.802521    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:39:54.802559 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.802530    2562 projected.go:196] Error preparing data for projected volume kube-api-access-fkcd4 for pod openshift-network-diagnostics/network-check-target-mf7bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:39:54.802622 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.802577    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4 podName:0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b nodeName:}" failed. No retries permitted until 2026-07-24 17:39:55.802560446 +0000 UTC m=+3.007252244 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkcd4" (UniqueName: "kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4") pod "network-check-target-mf7bb" (UID: "0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:39:54.903066 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.903043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:54.903153 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:54.903074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:54.903192 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.903168    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:39:54.903224 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.903219    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs podName:64665c61-db20-4439-a490-af90a1d0e579 nodeName:}" failed. No retries permitted until 2026-07-24 17:39:55.903206847 +0000 UTC m=+3.107898627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs") pod "network-metrics-daemon-kz48t" (UID: "64665c61-db20-4439-a490-af90a1d0e579") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:39:54.903282 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.903168    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:39:54.903329 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:54.903319    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret podName:74b9bf45-8bba-459e-bd93-ee92f990ae95 nodeName:}" failed. No retries permitted until 2026-07-24 17:39:55.903305479 +0000 UTC m=+3.107997275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret") pod "global-pull-secret-syncer-8fsnd" (UID: "74b9bf45-8bba-459e-bd93-ee92f990ae95") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:39:55.172195 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.172120    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:39:55.237472 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.237422    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:34:54 +0000 UTC" deadline="2028-04-03 22:21:14.727100188 +0000 UTC"
Jul 24 17:39:55.237472 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.237469    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14860h41m19.489635757s"
Jul 24 17:39:55.413253 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.413226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:55.413416 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.413363    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:39:55.442323 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.442209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pl7lv" event={"ID":"5a8021bc-a6d1-4d9b-be64-b04494e23e90","Type":"ContainerStarted","Data":"f2aa3d2e65e455fb21ef0c39f1556430618452fd64065e324bfdd9b00168f94a"}
Jul 24 17:39:55.448713 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.448681    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-pl7lv" podUID="5a8021bc-a6d1-4d9b-be64-b04494e23e90"
Jul 24 17:39:55.462340 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.462306    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tzghr" event={"ID":"c4707a45-429b-4d07-a7b9-41d3bcf570fa","Type":"ContainerStarted","Data":"eb294be7f127c635bb33753626d081de83d51d77c2ad060323545b34ec5dd383"}
Jul 24 17:39:55.479813 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.479762    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-95ljs" event={"ID":"bee228fc-b4ca-41fe-8bf2-f497792f028a","Type":"ContainerStarted","Data":"b9cea85d7b88a11719f679e20141dfeb62fc3ea73b1c366d6169a921bf2f3e4b"}
Jul 24 17:39:55.482179 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.482151    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t4rbd" event={"ID":"291c612c-f9bd-4525-a4b6-e3f6777def41","Type":"ContainerStarted","Data":"992ec8bb6c73cdfdcea18cce7632e12d705991354bb4a4e7f22d1b6daa9bf451"}
Jul 24 17:39:55.489790 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.489763    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t4t99" event={"ID":"81d447a3-e3df-4850-bcd4-60ef6e80cbcd","Type":"ContainerStarted","Data":"a76ed2ccb16991b0a8db4e36d884854e8e2f09f5b7f02a4a9466a6f90a7971cb"}
Jul 24 17:39:55.510163 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.510083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p9kck" event={"ID":"aec7a6de-7be2-41cc-b3f1-d447f0bbdab3","Type":"ContainerStarted","Data":"9a57e8acb96d29f5886134cc0d430313738b286a12ff5cedc22d19e6dd4d23d6"}
Jul 24 17:39:55.529606 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.529565    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal" event={"ID":"dd13083ddd3ee48db23d4b5c37fef9a4","Type":"ContainerStarted","Data":"aceca378fe6a6815b01bbc4e83a036dc61e7599af4c9ccc865bc81d7aa9a41a0"}
Jul 24 17:39:55.546093 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.545949    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq" event={"ID":"9248052a-97d7-41c5-9826-8485ec8dd504","Type":"ContainerStarted","Data":"10b81309d0fd74550a65ef32477096a8fb829292d79c139468c32bc0e66d4ee9"}
Jul 24 17:39:55.557395 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.557329    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nlgzt" event={"ID":"28f75bbf-9ea5-4984-ba60-2f464cda35fd","Type":"ContainerStarted","Data":"50859d54dfa0bec05e50f2da843010efcb86e976e50d26b51927edc5de77273d"}
Jul 24 17:39:55.585750 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.585718    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerStarted","Data":"0b9c019f18da422c372a29bef5c44ea0a5a47d7a300aa236abe071d447ecdaea"}
Jul 24 17:39:55.591426 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.591369    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal" event={"ID":"3e6877ce27ede56185f497d7314010a0","Type":"ContainerStarted","Data":"0e0945b95ddb55b7a841a6607cc3c5bf2db624329275065f7dd1bd80803f993a"}
Jul 24 17:39:55.811282 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.811199    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkcd4\" (UniqueName: \"kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4\") pod \"network-check-target-mf7bb\" (UID: \"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b\") " pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:39:55.811477 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.811465    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:39:55.811550 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.811488    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:39:55.811550 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.811500    2562 projected.go:196] Error preparing data for projected volume kube-api-access-fkcd4 for pod openshift-network-diagnostics/network-check-target-mf7bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:39:55.811661 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.811561    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4 podName:0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b nodeName:}" failed. No retries permitted until 2026-07-24 17:39:57.811539593 +0000 UTC m=+5.016231389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkcd4" (UniqueName: "kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4") pod "network-check-target-mf7bb" (UID: "0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:39:55.911798 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.911759    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:55.911997 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:55.911820    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:55.912082 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.911994    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:39:55.912138 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.912098    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret podName:74b9bf45-8bba-459e-bd93-ee92f990ae95 nodeName:}" failed. No retries permitted until 2026-07-24 17:39:57.912039607 +0000 UTC m=+5.116731384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret") pod "global-pull-secret-syncer-8fsnd" (UID: "74b9bf45-8bba-459e-bd93-ee92f990ae95") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:39:55.912570 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.912544    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:39:55.912680 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:55.912608    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs podName:64665c61-db20-4439-a490-af90a1d0e579 nodeName:}" failed. No retries permitted until 2026-07-24 17:39:57.912590436 +0000 UTC m=+5.117282212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs") pod "network-metrics-daemon-kz48t" (UID: "64665c61-db20-4439-a490-af90a1d0e579") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:39:56.238272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:56.238109    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:34:54 +0000 UTC" deadline="2028-01-08 16:07:26.617542621 +0000 UTC"
Jul 24 17:39:56.238272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:56.238178    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12790h27m30.379369108s"
Jul 24 17:39:56.402026 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:56.401494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:39:56.402026 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:56.401494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:56.402026 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:56.401640    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:39:56.402026 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:56.401753    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:39:56.726097 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:56.726063    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:39:57.409856 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:57.409374    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:57.409856 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:57.409515    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:39:57.827806 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:57.827694    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkcd4\" (UniqueName: \"kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4\") pod \"network-check-target-mf7bb\" (UID: \"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b\") " pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:39:57.828008 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:57.827905    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:39:57.828008 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:57.827931    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:39:57.828008 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:57.827945    2562 projected.go:196] Error preparing data for projected volume kube-api-access-fkcd4 for pod openshift-network-diagnostics/network-check-target-mf7bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:39:57.828008 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:57.828009    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4 podName:0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:01.827989697 +0000 UTC m=+9.032681481 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkcd4" (UniqueName: "kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4") pod "network-check-target-mf7bb" (UID: "0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:39:57.928968 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:57.928364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:57.928968 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:57.928412    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:57.928968 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:57.928549    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:39:57.928968 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:57.928587    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:39:57.928968 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:57.928614    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret podName:74b9bf45-8bba-459e-bd93-ee92f990ae95 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:01.928592085 +0000 UTC m=+9.133283888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret") pod "global-pull-secret-syncer-8fsnd" (UID: "74b9bf45-8bba-459e-bd93-ee92f990ae95") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:39:57.928968 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:57.928652    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs podName:64665c61-db20-4439-a490-af90a1d0e579 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:01.928633206 +0000 UTC m=+9.133324996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs") pod "network-metrics-daemon-kz48t" (UID: "64665c61-db20-4439-a490-af90a1d0e579") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:39:58.401841 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:58.401758    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:39:58.402055 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:58.401970    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:39:58.402055 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:58.402017    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:39:58.402189 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:58.402150    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:39:59.401725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:39:59.401688    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:39:59.402151 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:39:59.401828    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:00.401129 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:00.401093    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:00.401129 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:00.401122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:00.401342 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:00.401205    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:00.401396 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:00.401343    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:01.401304 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:01.401272    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:01.401767 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:01.401422    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:01.859131 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:01.859102    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkcd4\" (UniqueName: \"kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4\") pod \"network-check-target-mf7bb\" (UID: \"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b\") " pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:01.859313 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:01.859277    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:40:01.859313 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:01.859308    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:40:01.859486 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:01.859321    2562 projected.go:196] Error preparing data for projected volume kube-api-access-fkcd4 for pod openshift-network-diagnostics/network-check-target-mf7bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:40:01.859486 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:01.859382    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4 podName:0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:09.859363788 +0000 UTC m=+17.064055575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkcd4" (UniqueName: "kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4") pod "network-check-target-mf7bb" (UID: "0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:40:01.959624 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:01.959592    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:01.959624 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:01.959640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:01.959982 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:01.959776    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:40:01.959982 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:01.959777    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:40:01.959982 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:01.959832    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret podName:74b9bf45-8bba-459e-bd93-ee92f990ae95 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:09.959813226 +0000 UTC m=+17.164505014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret") pod "global-pull-secret-syncer-8fsnd" (UID: "74b9bf45-8bba-459e-bd93-ee92f990ae95") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:40:01.959982 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:01.959851    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs podName:64665c61-db20-4439-a490-af90a1d0e579 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:09.95984164 +0000 UTC m=+17.164533417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs") pod "network-metrics-daemon-kz48t" (UID: "64665c61-db20-4439-a490-af90a1d0e579") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:40:02.401738 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:02.401707    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:02.402216 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:02.401706    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:02.402216 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:02.401839    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:02.402216 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:02.401918    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:03.402166 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:03.402139    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:03.402545 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:03.402250    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:04.401934 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:04.401905    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:04.402097 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:04.401909    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:04.402097 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:04.402011    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:04.402178 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:04.402096    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:05.404360 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:05.404328    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:05.404811 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:05.404461    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:06.401732 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:06.401695    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:06.401935 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:06.401828    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:06.401935 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:06.401695    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:06.402061 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:06.401991    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:07.401267 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:07.401235    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:07.401667 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:07.401420    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:08.401551 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:08.401513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:08.402004 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:08.401659    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:08.402004 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:08.401707    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:08.402004 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:08.401825    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:09.401837 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.401805    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:09.402181 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:09.401959    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:09.649144 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.648715    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-95ljs" event={"ID":"bee228fc-b4ca-41fe-8bf2-f497792f028a","Type":"ContainerStarted","Data":"e4883869adad057923f5dc373852d0d34a683861b712eed27dae93bb79ec6833"}
Jul 24 17:40:09.655404 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.655379    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal" event={"ID":"dd13083ddd3ee48db23d4b5c37fef9a4","Type":"ContainerStarted","Data":"75e934e4a77e3a4caa6e99384f68dd37e77d4be32309fe83f27808c5752e707b"}
Jul 24 17:40:09.659152 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.659065    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:40:09.659940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.659905    2562 generic.go:356] "Generic (PLEG): container finished" podID="edf57894-9068-4294-952a-76003b90530f" containerID="4f222533b65b65ddaf9372d37295b4394da74a310619beb80ed9957e39ea7a69" exitCode=1
Jul 24 17:40:09.660020 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.659937    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerDied","Data":"4f222533b65b65ddaf9372d37295b4394da74a310619beb80ed9957e39ea7a69"}
Jul 24 17:40:09.660020 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.660015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerStarted","Data":"2837683f765e115ee4293cf0a1bf236b72a4855158fc52eb8784252c410ab0f0"}
Jul 24 17:40:09.670421 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.670358    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-95ljs" podStartSLOduration=1.020934902 podStartE2EDuration="15.670340466s" totalImagesPullingTime="14.649405564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:09.669563058 +0000 UTC m=+16.874254867" watchObservedRunningTime="2026-07-24 17:40:09.670340466 +0000 UTC m=+16.875032264"
Jul 24 17:40:09.691062 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.691016    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-89.ec2.internal" podStartSLOduration=15.690998889 podStartE2EDuration="15.690998889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:09.690916403 +0000 UTC m=+16.895608205" watchObservedRunningTime="2026-07-24 17:40:09.690998889 +0000 UTC m=+16.895690690"
Jul 24 17:40:09.918725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:09.918704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkcd4\" (UniqueName: \"kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4\") pod \"network-check-target-mf7bb\" (UID: \"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b\") " pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:09.918844 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:09.918827    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:40:09.918911 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:09.918849    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:40:09.918911 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:09.918861    2562 projected.go:196] Error preparing data for projected volume kube-api-access-fkcd4 for pod openshift-network-diagnostics/network-check-target-mf7bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:40:09.919016 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:09.918925    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4 podName:0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:25.918908811 +0000 UTC m=+33.123600589 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkcd4" (UniqueName: "kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4") pod "network-check-target-mf7bb" (UID: "0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:40:10.019326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.019272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:10.019326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.019302    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:10.019461 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:10.019388    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:40:10.019461 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:10.019399    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:40:10.019461 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:10.019430    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret podName:74b9bf45-8bba-459e-bd93-ee92f990ae95 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:26.019417337 +0000 UTC m=+33.224109115 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret") pod "global-pull-secret-syncer-8fsnd" (UID: "74b9bf45-8bba-459e-bd93-ee92f990ae95") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:40:10.019461 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:10.019442    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs podName:64665c61-db20-4439-a490-af90a1d0e579 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:26.019436291 +0000 UTC m=+33.224128066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs") pod "network-metrics-daemon-kz48t" (UID: "64665c61-db20-4439-a490-af90a1d0e579") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:40:10.401743 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.401712    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:10.401903 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.401818    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:10.401903 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:10.401831    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:10.402541 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:10.401944    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:10.615372 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.615237    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 17:40:10.662758 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.662732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t4t99" event={"ID":"81d447a3-e3df-4850-bcd4-60ef6e80cbcd","Type":"ContainerStarted","Data":"ab6e5af6b8bebd29a76bb3f092cc209b997f9b88322b74da61e3d0160d38c34a"}
Jul 24 17:40:10.663989 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.663968    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p9kck" event={"ID":"aec7a6de-7be2-41cc-b3f1-d447f0bbdab3","Type":"ContainerStarted","Data":"bacd236c43b81efc0ae2f15445370bdda04c1714323ebdf778f33fb31ee93db6"}
Jul 24 17:40:10.665582 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.665556    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq" event={"ID":"9248052a-97d7-41c5-9826-8485ec8dd504","Type":"ContainerStarted","Data":"94e004898082ff1f7a3b0702a61e137e70aec7bc733559ab4c460e0897eacf34"}
Jul 24 17:40:10.665674 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.665605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq" event={"ID":"9248052a-97d7-41c5-9826-8485ec8dd504","Type":"ContainerStarted","Data":"aa09b7d8e0c336d7f5e25b5241132efbf67518379b7de6b95f422fac530958af"}
Jul 24 17:40:10.666864 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.666843    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nlgzt" event={"ID":"28f75bbf-9ea5-4984-ba60-2f464cda35fd","Type":"ContainerStarted","Data":"302af9d10bdac5fc43689943f51aaed84e213f3053903f2d2cf3e66a962dc0f6"}
Jul 24 17:40:10.669146 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.669131    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:40:10.669467 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.669452    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerStarted","Data":"5ba3ade702b14de66577ea9aba83f9cf0ac79dfbd6c96dc02cfbcc4adadf8c03"}
Jul 24 17:40:10.669519 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.669501    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerStarted","Data":"b3d958ad7430f9ce86f631a56558cef63516f378462a9f48cf58d428649bf2ce"}
Jul 24 17:40:10.669555 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.669534    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerStarted","Data":"abd623c230668dfebd3cc460b4e14b9ec9e452b577ba4d354d8c855e248d37af"}
Jul 24 17:40:10.669594 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.669576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerStarted","Data":"78b41b49c5685e97a374cdae33e2795cbdb50c33fb851f4e0d8668b70ca3e1ec"}
Jul 24 17:40:10.670649 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.670631    2562 generic.go:356] "Generic (PLEG): container finished" podID="3e6877ce27ede56185f497d7314010a0" containerID="dc80796a859cbb5374c14b2b6bd3cf73e08bb9be2121f4a92a0a4cba94b3c635" exitCode=0
Jul 24 17:40:10.670723 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.670686    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal" event={"ID":"3e6877ce27ede56185f497d7314010a0","Type":"ContainerDied","Data":"dc80796a859cbb5374c14b2b6bd3cf73e08bb9be2121f4a92a0a4cba94b3c635"}
Jul 24 17:40:10.672089 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.672067    2562 generic.go:356] "Generic (PLEG): container finished" podID="c4707a45-429b-4d07-a7b9-41d3bcf570fa" containerID="36d030a7ca240bfafd8a9819a59170628b0fb94e7aa226d5c9292485bec4894a" exitCode=0
Jul 24 17:40:10.672163 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.672142    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tzghr" event={"ID":"c4707a45-429b-4d07-a7b9-41d3bcf570fa","Type":"ContainerDied","Data":"36d030a7ca240bfafd8a9819a59170628b0fb94e7aa226d5c9292485bec4894a"}
Jul 24 17:40:10.673497 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.673478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t4rbd" event={"ID":"291c612c-f9bd-4525-a4b6-e3f6777def41","Type":"ContainerStarted","Data":"ec75738e280af4d2477e1400f088870daf8cb7cf44ac9eb1836dc8a6c9e8098f"}
Jul 24 17:40:10.676055 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.676013    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t4t99" podStartSLOduration=2.986030547 podStartE2EDuration="17.675997224s" totalImagesPullingTime="14.689966677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:10.675275611 +0000 UTC m=+17.879967409" watchObservedRunningTime="2026-07-24 17:40:10.675997224 +0000 UTC m=+17.880689019"
Jul 24 17:40:10.703307 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.703267    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p9kck" podStartSLOduration=3.37938174 podStartE2EDuration="17.703251227s" totalImagesPullingTime="14.323869487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:10.702694232 +0000 UTC m=+17.907386028" watchObservedRunningTime="2026-07-24 17:40:10.703251227 +0000 UTC m=+17.907943030"
Jul 24 17:40:10.719958 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.719925    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t4rbd" podStartSLOduration=3.030651704 podStartE2EDuration="17.719913013s" totalImagesPullingTime="14.689261309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:10.71938618 +0000 UTC m=+17.924077979" watchObservedRunningTime="2026-07-24 17:40:10.719913013 +0000 UTC m=+17.924604810"
Jul 24 17:40:10.734596 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:10.734554    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nlgzt" podStartSLOduration=2.049556841 podStartE2EDuration="16.734542569s" totalImagesPullingTime="14.684985728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:10.734319352 +0000 UTC m=+17.939011146" watchObservedRunningTime="2026-07-24 17:40:10.734542569 +0000 UTC m=+17.939234365"
Jul 24 17:40:11.389536 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:11.389424    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T17:40:10.615369028Z","UUID":"9add6b6b-101e-4eca-b3f8-19fd79705e42","Handler":null,"Name":"","Endpoint":""}
Jul 24 17:40:11.391486 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:11.391467    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 17:40:11.391615 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:11.391501    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 17:40:11.405281 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:11.404847    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:11.405281 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:11.404995    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:11.680688 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:11.680650    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal" event={"ID":"3e6877ce27ede56185f497d7314010a0","Type":"ContainerStarted","Data":"05a366c44dc9bc0ec4d5af8d896d5caf47acd20dec06638a415ad2ccf7551297"}
Jul 24 17:40:11.702989 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:11.702938    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-89.ec2.internal" podStartSLOduration=17.702918548 podStartE2EDuration="17.702918548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:11.702662916 +0000 UTC m=+18.907354714" watchObservedRunningTime="2026-07-24 17:40:11.702918548 +0000 UTC m=+18.907610346"
Jul 24 17:40:11.955311 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:11.955241    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:40:11.956043 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:11.956021    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:40:12.401277 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:12.401248    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:12.401461 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:12.401249    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:12.401461 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:12.401364    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:12.401570 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:12.401475    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:12.687601 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:12.687524    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq" event={"ID":"9248052a-97d7-41c5-9826-8485ec8dd504","Type":"ContainerStarted","Data":"e624b212f0dcd4d43ca20e01f693675e898f8a2874beae0450753255c7f29606"}
Jul 24 17:40:12.691468 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:12.691444    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:40:12.691975 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:12.691892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerStarted","Data":"f23dd6ce79f487167b4825c57857549465d177546806774bcc47ff56b0e2ba26"}
Jul 24 17:40:12.692253 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:12.692200    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:40:12.692702 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:12.692630    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p9kck"
Jul 24 17:40:12.706185 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:12.706152    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ncrq" podStartSLOduration=2.280770959 podStartE2EDuration="18.706138993s" totalImagesPullingTime="16.425368034s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:12.705992117 +0000 UTC m=+19.910683915" watchObservedRunningTime="2026-07-24 17:40:12.706138993 +0000 UTC m=+19.910830792"
Jul 24 17:40:13.402167 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:13.402136    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:13.402364 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:13.402255    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:14.401231 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:14.401033    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:14.401687 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:14.401039    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:14.401687 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:14.401285    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:14.401687 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:14.401400    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:14.700577 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:14.700510    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:40:14.700959 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:14.700923    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerStarted","Data":"b660e754505454e867c37814bd41daa1dbc6d6b2f72c62ae76e2664a04c7a0f9"}
Jul 24 17:40:14.701453 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:14.701436    2562 scope.go:117] "RemoveContainer" containerID="4f222533b65b65ddaf9372d37295b4394da74a310619beb80ed9957e39ea7a69"
Jul 24 17:40:15.401058 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:15.401026    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:15.401218 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:15.401171    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:16.147493 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.147466    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mf7bb"]
Jul 24 17:40:16.148144 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.147579    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:16.148144 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:16.147668    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:16.152777 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.152748    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8fsnd"]
Jul 24 17:40:16.153190 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.152803    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kz48t"]
Jul 24 17:40:16.153190 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.152872    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:16.153190 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:16.152992    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:16.153190 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.153052    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:16.153190 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:16.153123    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:16.707598 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.707562    2562 generic.go:356] "Generic (PLEG): container finished" podID="c4707a45-429b-4d07-a7b9-41d3bcf570fa" containerID="5d41610487fe7491a938f2600954dd28b2675631cd802a4a5706a1241a0868bd" exitCode=0
Jul 24 17:40:16.707854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.707624    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tzghr" event={"ID":"c4707a45-429b-4d07-a7b9-41d3bcf570fa","Type":"ContainerDied","Data":"5d41610487fe7491a938f2600954dd28b2675631cd802a4a5706a1241a0868bd"}
Jul 24 17:40:16.712033 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.712014    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:40:16.712397 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.712370    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" event={"ID":"edf57894-9068-4294-952a-76003b90530f","Type":"ContainerStarted","Data":"5d19b078e958bd7dd006ad01e0db024e77e41c91673a6e27d327726c36370322"}
Jul 24 17:40:16.712518 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.712503    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 17:40:16.712873 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.712745    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:40:16.712873 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.712819    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:40:16.729512 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.729459    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:40:16.729772 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.729755    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:40:16.765677 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.765625    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv" podStartSLOduration=8.817888634 podStartE2EDuration="23.765606829s" totalImagesPullingTime="14.947718195s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:16.763584065 +0000 UTC m=+23.968275861" watchObservedRunningTime="2026-07-24 17:40:16.765606829 +0000 UTC m=+23.970298627"
Jul 24 17:40:16.941480 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:16.941456    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:40:17.405163 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:17.405133    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:17.405862 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:17.405133    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:17.405862 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:17.405240    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:17.405862 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:17.405293    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:18.401911 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:18.401822    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:18.402075 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:18.401991    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:19.405026 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:19.404812    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:19.405472 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:19.404894    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:19.405472 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:19.405124    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:19.405472 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:19.405207    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:20.401500 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:20.401473    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:20.401692 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:20.401647    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:21.404642 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:21.404615    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:21.404642 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:21.404635    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:21.405090 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:21.404737    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mf7bb" podUID="0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b"
Jul 24 17:40:21.405090 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:21.404840    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8fsnd" podUID="74b9bf45-8bba-459e-bd93-ee92f990ae95"
Jul 24 17:40:21.725376 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:21.725333    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pl7lv" event={"ID":"5a8021bc-a6d1-4d9b-be64-b04494e23e90","Type":"ContainerStarted","Data":"6d2a31fad11b02546302026437a3254f72c58e6063f940682e0fe290de8d43d1"}
Jul 24 17:40:21.727537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:21.727512    2562 generic.go:356] "Generic (PLEG): container finished" podID="c4707a45-429b-4d07-a7b9-41d3bcf570fa" containerID="f270510285f11a8623fe0dea1b7cc7e3fed965a4beee6a17595ce0c808985e00" exitCode=0
Jul 24 17:40:21.727637 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:21.727570    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tzghr" event={"ID":"c4707a45-429b-4d07-a7b9-41d3bcf570fa","Type":"ContainerDied","Data":"f270510285f11a8623fe0dea1b7cc7e3fed965a4beee6a17595ce0c808985e00"}
Jul 24 17:40:21.761056 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:21.760943    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pl7lv" podStartSLOduration=1.05184202 podStartE2EDuration="27.760928539s" totalImagesPullingTime="26.709086519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-24 17:40:21.742011502 +0000 UTC m=+28.946703299" watchObservedRunningTime="2026-07-24 17:40:21.760928539 +0000 UTC m=+28.965620335"
Jul 24 17:40:22.401378 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:22.401338    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:22.401555 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:22.401454    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kz48t" podUID="64665c61-db20-4439-a490-af90a1d0e579"
Jul 24 17:40:23.084288 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.084121    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-89.ec2.internal" event="NodeReady"
Jul 24 17:40:23.084694 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.084388    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 17:40:23.133455 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.133428    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55d8dbcb57-tglbk"]
Jul 24 17:40:23.147772 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.147756    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7jczg"]
Jul 24 17:40:23.147903 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.147871    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.150026 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.150003    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 17:40:23.150026 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.150023    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-crf7b\""
Jul 24 17:40:23.150147 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.150030    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 17:40:23.150455 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.150438    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 17:40:23.156336 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.156318    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 17:40:23.162983 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.162956    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55d8dbcb57-tglbk"]
Jul 24 17:40:23.163054 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.162998    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v622r"]
Jul 24 17:40:23.163116 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.162996    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.165516 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.165501    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tq4qz\""
Jul 24 17:40:23.165683 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.165672    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 17:40:23.165831 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.165820    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 17:40:23.180462 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.180443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7jczg"]
Jul 24 17:40:23.180533 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.180484    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v622r"]
Jul 24 17:40:23.180571 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.180564    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v622r"
Jul 24 17:40:23.182609 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.182590    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 17:40:23.182693 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.182610    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 17:40:23.182693 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.182594    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 17:40:23.183013 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.182999    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jpmgf\""
Jul 24 17:40:23.315793 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.315771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p9kck_aec7a6de-7be2-41cc-b3f1-d447f0bbdab3/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:23.323738 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.323715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93ed42c-d048-484d-941a-93193f1e13cd-tmp-dir\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.323831 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.323748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9296j\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-kube-api-access-9296j\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.323831 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.323798    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.323949 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.323839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d93ed42c-d048-484d-941a-93193f1e13cd-config-volume\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.323949 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.323898    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78ptl\" (UniqueName: \"kubernetes.io/projected/d93ed42c-d048-484d-941a-93193f1e13cd-kube-api-access-78ptl\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.323949 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.323927    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-image-registry-private-configuration\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.324067 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.323954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6011f48-0283-430a-8d6a-e81ce31eec5b-ca-trust-extracted\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.324067 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.323971    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-certificates\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.324067 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.323985    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc2d1617-bb81-4541-9971-09854abe6150-cert\") pod \"ingress-canary-v622r\" (UID: \"cc2d1617-bb81-4541-9971-09854abe6150\") " pod="openshift-ingress-canary/ingress-canary-v622r"
Jul 24 17:40:23.324067 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.324005    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bf5l\" (UniqueName: \"kubernetes.io/projected/cc2d1617-bb81-4541-9971-09854abe6150-kube-api-access-6bf5l\") pod \"ingress-canary-v622r\" (UID: \"cc2d1617-bb81-4541-9971-09854abe6150\") " pod="openshift-ingress-canary/ingress-canary-v622r"
Jul 24 17:40:23.324067 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.324048    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.324067 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.324065    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-installation-pull-secrets\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.324254 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.324085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-bound-sa-token\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.324254 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.324137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-trusted-ca\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.405725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.405668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p9kck_aec7a6de-7be2-41cc-b3f1-d447f0bbdab3/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:23.406489 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.406474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:23.406673 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.406654    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:23.408731 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.408710    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 17:40:23.408829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.408770    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 17:40:23.408912 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.408828    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t8nmh\""
Jul 24 17:40:23.408912 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.408848    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 17:40:23.424347 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424329    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.424439 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-installation-pull-secrets\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.424439 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-bound-sa-token\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.424554 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.424449    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:40:23.424554 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.424465    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d8dbcb57-tglbk: secret "image-registry-tls" not found
Jul 24 17:40:23.424554 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-trusted-ca\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.424554 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.424523    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls podName:e6011f48-0283-430a-8d6a-e81ce31eec5b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:23.924504579 +0000 UTC m=+31.129196368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls") pod "image-registry-55d8dbcb57-tglbk" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b") : secret "image-registry-tls" not found
Jul 24 17:40:23.424759 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424561    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93ed42c-d048-484d-941a-93193f1e13cd-tmp-dir\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.424759 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424591    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9296j\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-kube-api-access-9296j\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.424759 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.424759 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d93ed42c-d048-484d-941a-93193f1e13cd-config-volume\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.424967 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.424758    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:40:23.424967 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.424831    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls podName:d93ed42c-d048-484d-941a-93193f1e13cd nodeName:}" failed. No retries permitted until 2026-07-24 17:40:23.924815132 +0000 UTC m=+31.129506914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls") pod "dns-default-7jczg" (UID: "d93ed42c-d048-484d-941a-93193f1e13cd") : secret "dns-default-metrics-tls" not found
Jul 24 17:40:23.424967 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424846    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78ptl\" (UniqueName: \"kubernetes.io/projected/d93ed42c-d048-484d-941a-93193f1e13cd-kube-api-access-78ptl\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.424967 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-image-registry-private-configuration\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.424967 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424911    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6011f48-0283-430a-8d6a-e81ce31eec5b-ca-trust-extracted\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.424967 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-certificates\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.424967 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc2d1617-bb81-4541-9971-09854abe6150-cert\") pod \"ingress-canary-v622r\" (UID: \"cc2d1617-bb81-4541-9971-09854abe6150\") " pod="openshift-ingress-canary/ingress-canary-v622r"
Jul 24 17:40:23.425309 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93ed42c-d048-484d-941a-93193f1e13cd-tmp-dir\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.425309 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.424995    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf5l\" (UniqueName: \"kubernetes.io/projected/cc2d1617-bb81-4541-9971-09854abe6150-kube-api-access-6bf5l\") pod \"ingress-canary-v622r\" (UID: \"cc2d1617-bb81-4541-9971-09854abe6150\") " pod="openshift-ingress-canary/ingress-canary-v622r"
Jul 24 17:40:23.425309 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.425233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d93ed42c-d048-484d-941a-93193f1e13cd-config-volume\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.425479 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.425377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6011f48-0283-430a-8d6a-e81ce31eec5b-ca-trust-extracted\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.425479 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.425389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-trusted-ca\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.425820 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.425682    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-certificates\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.428352 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.428332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-installation-pull-secrets\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.428352 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.428350    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-image-registry-private-configuration\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.428510 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.428405    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc2d1617-bb81-4541-9971-09854abe6150-cert\") pod \"ingress-canary-v622r\" (UID: \"cc2d1617-bb81-4541-9971-09854abe6150\") " pod="openshift-ingress-canary/ingress-canary-v622r"
Jul 24 17:40:23.432827 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.432792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-bound-sa-token\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.432988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.432966    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9296j\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-kube-api-access-9296j\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.433213 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.433171    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78ptl\" (UniqueName: \"kubernetes.io/projected/d93ed42c-d048-484d-941a-93193f1e13cd-kube-api-access-78ptl\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.433554 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.433526    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bf5l\" (UniqueName: \"kubernetes.io/projected/cc2d1617-bb81-4541-9971-09854abe6150-kube-api-access-6bf5l\") pod \"ingress-canary-v622r\" (UID: \"cc2d1617-bb81-4541-9971-09854abe6150\") " pod="openshift-ingress-canary/ingress-canary-v622r"
Jul 24 17:40:23.488773 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.488756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v622r"
Jul 24 17:40:23.513584 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.513550    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p9kck_aec7a6de-7be2-41cc-b3f1-d447f0bbdab3/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:23.568624 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.568583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p9kck_aec7a6de-7be2-41cc-b3f1-d447f0bbdab3/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:23.719799 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.719771    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v622r"]
Jul 24 17:40:23.722948 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:23.722925    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc2d1617_bb81_4541_9971_09854abe6150.slice/crio-036aa879e7634a8929ef26bff2fb865ab599300ee6a5efae0930117e4d08ff70 WatchSource:0}: Error finding container 036aa879e7634a8929ef26bff2fb865ab599300ee6a5efae0930117e4d08ff70: Status 404 returned error can't find the container with id 036aa879e7634a8929ef26bff2fb865ab599300ee6a5efae0930117e4d08ff70
Jul 24 17:40:23.733953 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.733927    2562 generic.go:356] "Generic (PLEG): container finished" podID="c4707a45-429b-4d07-a7b9-41d3bcf570fa" containerID="a89760c4259255e77c4877a17e185c150e8041c4cfba0e1777257288313968f0" exitCode=0
Jul 24 17:40:23.734029 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.734003    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tzghr" event={"ID":"c4707a45-429b-4d07-a7b9-41d3bcf570fa","Type":"ContainerDied","Data":"a89760c4259255e77c4877a17e185c150e8041c4cfba0e1777257288313968f0"}
Jul 24 17:40:23.735116 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.735093    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v622r" event={"ID":"cc2d1617-bb81-4541-9971-09854abe6150","Type":"ContainerStarted","Data":"036aa879e7634a8929ef26bff2fb865ab599300ee6a5efae0930117e4d08ff70"}
Jul 24 17:40:23.934505 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.934456    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:23.934587 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:23.934516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:23.934626 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.934592    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:40:23.934626 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.934602    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:40:23.934626 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.934618    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d8dbcb57-tglbk: secret "image-registry-tls" not found
Jul 24 17:40:23.934711 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.934641    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls podName:d93ed42c-d048-484d-941a-93193f1e13cd nodeName:}" failed. No retries permitted until 2026-07-24 17:40:24.934623901 +0000 UTC m=+32.139315677 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls") pod "dns-default-7jczg" (UID: "d93ed42c-d048-484d-941a-93193f1e13cd") : secret "dns-default-metrics-tls" not found
Jul 24 17:40:23.934711 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:23.934654    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls podName:e6011f48-0283-430a-8d6a-e81ce31eec5b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:24.934648448 +0000 UTC m=+32.139340223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls") pod "image-registry-55d8dbcb57-tglbk" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b") : secret "image-registry-tls" not found
Jul 24 17:40:24.401671 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:24.401644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:24.404357 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:24.404138    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 17:40:24.404357 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:24.404166    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-62mvz\""
Jul 24 17:40:24.941974 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:24.941936    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:24.942191 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:24.942012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:24.942191 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:24.942100    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:40:24.942191 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:24.942123    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:40:24.942191 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:24.942134    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d8dbcb57-tglbk: secret "image-registry-tls" not found
Jul 24 17:40:24.942191 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:24.942166    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls podName:d93ed42c-d048-484d-941a-93193f1e13cd nodeName:}" failed. No retries permitted until 2026-07-24 17:40:26.942148077 +0000 UTC m=+34.146839868 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls") pod "dns-default-7jczg" (UID: "d93ed42c-d048-484d-941a-93193f1e13cd") : secret "dns-default-metrics-tls" not found
Jul 24 17:40:24.942191 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:24.942185    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls podName:e6011f48-0283-430a-8d6a-e81ce31eec5b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:26.942175315 +0000 UTC m=+34.146867089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls") pod "image-registry-55d8dbcb57-tglbk" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b") : secret "image-registry-tls" not found
Jul 24 17:40:25.950141 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:25.949844    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkcd4\" (UniqueName: \"kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4\") pod \"network-check-target-mf7bb\" (UID: \"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b\") " pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:25.954020 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:25.953842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkcd4\" (UniqueName: \"kubernetes.io/projected/0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b-kube-api-access-fkcd4\") pod \"network-check-target-mf7bb\" (UID: \"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b\") " pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:26.050668 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.050637    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:26.050815 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.050689    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:26.051236 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:26.051216    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 17:40:26.051321 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:26.051290    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs podName:64665c61-db20-4439-a490-af90a1d0e579 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:58.051271348 +0000 UTC m=+65.255963126 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs") pod "network-metrics-daemon-kz48t" (UID: "64665c61-db20-4439-a490-af90a1d0e579") : secret "metrics-daemon-secret" not found
Jul 24 17:40:26.053306 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.053282    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74b9bf45-8bba-459e-bd93-ee92f990ae95-original-pull-secret\") pod \"global-pull-secret-syncer-8fsnd\" (UID: \"74b9bf45-8bba-459e-bd93-ee92f990ae95\") " pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:26.115414 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.115390    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:26.120584 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.120531    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8fsnd"
Jul 24 17:40:26.704817 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.704706    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mf7bb"]
Jul 24 17:40:26.707434 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.707411    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8fsnd"]
Jul 24 17:40:26.708352 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:26.708317    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad9ca8c_7fdf_4e9e_bb7f_4bd3c962127b.slice/crio-ba627903f92d20d83f927890b60fd76f1a0dae69415f3ba0f903995ede974f12 WatchSource:0}: Error finding container ba627903f92d20d83f927890b60fd76f1a0dae69415f3ba0f903995ede974f12: Status 404 returned error can't find the container with id ba627903f92d20d83f927890b60fd76f1a0dae69415f3ba0f903995ede974f12
Jul 24 17:40:26.710275 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:26.710251    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b9bf45_8bba_459e_bd93_ee92f990ae95.slice/crio-b7e22efea48162edc1f019534421b000d5c4f93951eb6dd76e2836e4d662187d WatchSource:0}: Error finding container b7e22efea48162edc1f019534421b000d5c4f93951eb6dd76e2836e4d662187d: Status 404 returned error can't find the container with id b7e22efea48162edc1f019534421b000d5c4f93951eb6dd76e2836e4d662187d
Jul 24 17:40:26.743353 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.743329    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8fsnd" event={"ID":"74b9bf45-8bba-459e-bd93-ee92f990ae95","Type":"ContainerStarted","Data":"b7e22efea48162edc1f019534421b000d5c4f93951eb6dd76e2836e4d662187d"}
Jul 24 17:40:26.744276 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.744255    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mf7bb" event={"ID":"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b","Type":"ContainerStarted","Data":"ba627903f92d20d83f927890b60fd76f1a0dae69415f3ba0f903995ede974f12"}
Jul 24 17:40:26.957778 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.957708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:26.958235 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:26.957780    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:26.958235 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:26.957866    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:40:26.958235 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:26.957911    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:40:26.958235 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:26.957923    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d8dbcb57-tglbk: secret "image-registry-tls" not found
Jul 24 17:40:26.958235 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:26.957953    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls podName:d93ed42c-d048-484d-941a-93193f1e13cd nodeName:}" failed. No retries permitted until 2026-07-24 17:40:30.957932805 +0000 UTC m=+38.162624603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls") pod "dns-default-7jczg" (UID: "d93ed42c-d048-484d-941a-93193f1e13cd") : secret "dns-default-metrics-tls" not found
Jul 24 17:40:26.958235 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:26.957968    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls podName:e6011f48-0283-430a-8d6a-e81ce31eec5b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:30.957961835 +0000 UTC m=+38.162653610 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls") pod "image-registry-55d8dbcb57-tglbk" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b") : secret "image-registry-tls" not found
Jul 24 17:40:27.510397 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.510256    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"]
Jul 24 17:40:27.535801 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.535778    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"]
Jul 24 17:40:27.535959 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.535923    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.538318 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.538283    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 24 17:40:27.538453 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.538415    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 24 17:40:27.538453 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.538444    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:40:27.538552 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.538492    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-n58qc\""
Jul 24 17:40:27.539254 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.539235    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 24 17:40:27.555985 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.555959    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-m7799"]
Jul 24 17:40:27.556099 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.556084    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:27.558408 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.558385    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 17:40:27.558519 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.558430    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 24 17:40:27.559214 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.559193    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 24 17:40:27.559343 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.559234    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 17:40:27.559343 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.559247    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wm84m\""
Jul 24 17:40:27.569855 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.569838    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5cb6869597-2kxmd"]
Jul 24 17:40:27.569977 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.569962    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.572230 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.572214    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 24 17:40:27.572325 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.572308    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-q75lx\""
Jul 24 17:40:27.572542 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.572526    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 24 17:40:27.572631 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.572615    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 17:40:27.572952 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.572938    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 17:40:27.581636 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.581606    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 24 17:40:27.590135 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.590116    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627"]
Jul 24 17:40:27.590269 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.590250    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.593035 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.593018    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c2x66\""
Jul 24 17:40:27.593131 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.593092    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 24 17:40:27.593131 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.593101    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 24 17:40:27.593131 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.593099    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 24 17:40:27.593282 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.593184    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 17:40:27.593384 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.593356    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 17:40:27.593451 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.593431    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 24 17:40:27.610033 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.610016    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"]
Jul 24 17:40:27.610124 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.610048    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"]
Jul 24 17:40:27.610124 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.610117    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627"
Jul 24 17:40:27.610246 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.610138    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-m7799"]
Jul 24 17:40:27.610246 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.610151    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5cb6869597-2kxmd"]
Jul 24 17:40:27.610246 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.610164    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627"]
Jul 24 17:40:27.610246 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.610196    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"]
Jul 24 17:40:27.612934 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.612917    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:40:27.613015 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.612972    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-dzq6j\""
Jul 24 17:40:27.613545 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.613526    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 24 17:40:27.630243 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.630225    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-8gfzp"]
Jul 24 17:40:27.630341 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.630323    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:27.632893 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.632715    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 24 17:40:27.632893 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.632742    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 24 17:40:27.632893 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.632761    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:40:27.632893 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.632820    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wq865\""
Jul 24 17:40:27.644426 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.644405    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"]
Jul 24 17:40:27.644536 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.644447    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-8gfzp"]
Jul 24 17:40:27.644536 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.644519    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.646765 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.646743    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 24 17:40:27.647321 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.647180    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 24 17:40:27.647321 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.647196    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zj5jx\""
Jul 24 17:40:27.647552 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.647527    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 24 17:40:27.647672 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.647655    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:40:27.653493 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.653463    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 24 17:40:27.662845 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.662823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.662946 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.662865    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-service-ca-bundle\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.662946 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.662908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65571cc3-f16a-4cfa-a73c-50380be899c2-serving-cert\") pod \"service-ca-operator-5d6c75f84f-nblq2\" (UID: \"65571cc3-f16a-4cfa-a73c-50380be899c2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.662946 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.662937    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:27.663085 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.663011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbn5q\" (UniqueName: \"kubernetes.io/projected/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-kube-api-access-nbn5q\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:27.663085 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.663046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qxcp\" (UniqueName: \"kubernetes.io/projected/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-kube-api-access-6qxcp\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.663197 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.663095    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-tmp\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.663197 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.663131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w54v\" (UniqueName: \"kubernetes.io/projected/65571cc3-f16a-4cfa-a73c-50380be899c2-kube-api-access-8w54v\") pod \"service-ca-operator-5d6c75f84f-nblq2\" (UID: \"65571cc3-f16a-4cfa-a73c-50380be899c2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.663300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.663203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:27.663300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.663238    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65571cc3-f16a-4cfa-a73c-50380be899c2-config\") pod \"service-ca-operator-5d6c75f84f-nblq2\" (UID: \"65571cc3-f16a-4cfa-a73c-50380be899c2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.663300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.663266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-snapshots\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.663300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.663290    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-serving-cert\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.691981 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.691960    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"]
Jul 24 17:40:27.713528 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.713499    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct"]
Jul 24 17:40:27.713673 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.713651    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.715730 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.715713    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 24 17:40:27.716004 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.715972    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 24 17:40:27.716106 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.716054    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 24 17:40:27.716106 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.716075    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-cq7rb\""
Jul 24 17:40:27.716226 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.716057    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:40:27.727711 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.727688    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"]
Jul 24 17:40:27.727790 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.727722    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct"]
Jul 24 17:40:27.727847 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.727817    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct"
Jul 24 17:40:27.730038 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.730023    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-257ls\""
Jul 24 17:40:27.748506 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.748482    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v622r" event={"ID":"cc2d1617-bb81-4541-9971-09854abe6150","Type":"ContainerStarted","Data":"ab96bbd64e1d064471246b5d2a5857cd980bf70ab2b888df81d67e4e0e131b71"}
Jul 24 17:40:27.763145 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.763071    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v622r" podStartSLOduration=1.842772379 podStartE2EDuration="4.763054549s" totalImagesPullingTime="2.92028217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:27.762443498 +0000 UTC m=+34.967135306" watchObservedRunningTime="2026-07-24 17:40:27.763054549 +0000 UTC m=+34.967746345"
Jul 24 17:40:27.763865 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.763844    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-snapshots\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.763977 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.763892    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-serving-cert\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.763977 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.763921    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/556df0ae-3850-45c5-a889-d36921adf671-trusted-ca\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.763977 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.763951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsmlr\" (UniqueName: \"kubernetes.io/projected/556df0ae-3850-45c5-a889-d36921adf671-kube-api-access-wsmlr\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.764115 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.763997    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh5lv\" (UniqueName: \"kubernetes.io/projected/b3cb69d8-c6ec-4537-aae4-d202504f2fc8-kube-api-access-hh5lv\") pod \"volume-data-source-validator-549c6d54cf-db627\" (UID: \"b3cb69d8-c6ec-4537-aae4-d202504f2fc8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627"
Jul 24 17:40:27.764115 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8bnq\" (UniqueName: \"kubernetes.io/projected/b3bc8432-2ec9-46b9-891f-70c53f921b38-kube-api-access-g8bnq\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:27.764222 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.764222 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65571cc3-f16a-4cfa-a73c-50380be899c2-serving-cert\") pod \"service-ca-operator-5d6c75f84f-nblq2\" (UID: \"65571cc3-f16a-4cfa-a73c-50380be899c2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.764222 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-service-ca-bundle\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.764363 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.764363 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qxcp\" (UniqueName: \"kubernetes.io/projected/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-kube-api-access-6qxcp\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.764363 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2hsk\" (UniqueName: \"kubernetes.io/projected/2f5568f4-e4a5-4007-adb7-780b43c081ad-kube-api-access-v2hsk\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.764363 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w54v\" (UniqueName: \"kubernetes.io/projected/65571cc3-f16a-4cfa-a73c-50380be899c2-kube-api-access-8w54v\") pod \"service-ca-operator-5d6c75f84f-nblq2\" (UID: \"65571cc3-f16a-4cfa-a73c-50380be899c2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.764363 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764319    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-tmp\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.764363 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65571cc3-f16a-4cfa-a73c-50380be899c2-config\") pod \"service-ca-operator-5d6c75f84f-nblq2\" (UID: \"65571cc3-f16a-4cfa-a73c-50380be899c2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/556df0ae-3850-45c5-a889-d36921adf671-config\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764412    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/556df0ae-3850-45c5-a889-d36921adf671-serving-cert\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764457    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764486    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-default-certificate\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbn5q\" (UniqueName: \"kubernetes.io/projected/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-kube-api-access-nbn5q\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764537    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764559    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764582    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-stats-auth\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.764645 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.764618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-snapshots\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.765122 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:27.764717    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:27.765122 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:27.764783    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls podName:a9411acd-1297-42d0-8b2e-bc84a5cfec3b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:28.264765575 +0000 UTC m=+35.469457352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kzcs4" (UID: "a9411acd-1297-42d0-8b2e-bc84a5cfec3b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:27.766401 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.765131    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65571cc3-f16a-4cfa-a73c-50380be899c2-config\") pod \"service-ca-operator-5d6c75f84f-nblq2\" (UID: \"65571cc3-f16a-4cfa-a73c-50380be899c2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.766401 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.765378    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:27.766401 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.765793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-tmp\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.766401 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.765919    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.766401 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.766308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-service-ca-bundle\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.768326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.768273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65571cc3-f16a-4cfa-a73c-50380be899c2-serving-cert\") pod \"service-ca-operator-5d6c75f84f-nblq2\" (UID: \"65571cc3-f16a-4cfa-a73c-50380be899c2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.768417 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.768365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-serving-cert\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.777676 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.777650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbn5q\" (UniqueName: \"kubernetes.io/projected/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-kube-api-access-nbn5q\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:27.779193 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.779168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qxcp\" (UniqueName: \"kubernetes.io/projected/d66bc6dd-82ce-473f-8aba-de7d3929b6bb-kube-api-access-6qxcp\") pod \"insights-operator-5885fdb759-m7799\" (UID: \"d66bc6dd-82ce-473f-8aba-de7d3929b6bb\") " pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.779288 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.779197    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w54v\" (UniqueName: \"kubernetes.io/projected/65571cc3-f16a-4cfa-a73c-50380be899c2-kube-api-access-8w54v\") pod \"service-ca-operator-5d6c75f84f-nblq2\" (UID: \"65571cc3-f16a-4cfa-a73c-50380be899c2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.847118 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.847077    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"
Jul 24 17:40:27.865238 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2hsk\" (UniqueName: \"kubernetes.io/projected/2f5568f4-e4a5-4007-adb7-780b43c081ad-kube-api-access-v2hsk\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.865355 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8dd06a5f-d5af-4e05-bd87-afdbf92c7b43-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q\" (UID: \"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.865355 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/556df0ae-3850-45c5-a889-d36921adf671-config\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.865355 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8dd06a5f-d5af-4e05-bd87-afdbf92c7b43-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q\" (UID: \"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.865497 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/556df0ae-3850-45c5-a889-d36921adf671-serving-cert\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.865497 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-default-certificate\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.865497 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865449    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hnxf\" (UniqueName: \"kubernetes.io/projected/8dd06a5f-d5af-4e05-bd87-afdbf92c7b43-kube-api-access-8hnxf\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q\" (UID: \"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.865497 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865479    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.865669 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:27.865669 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-stats-auth\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.865669 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/556df0ae-3850-45c5-a889-d36921adf671-trusted-ca\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.865669 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865623    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsmlr\" (UniqueName: \"kubernetes.io/projected/556df0ae-3850-45c5-a889-d36921adf671-kube-api-access-wsmlr\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.865669 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:27.865647    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:28.365626442 +0000 UTC m=+35.570318232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:27.866006 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh5lv\" (UniqueName: \"kubernetes.io/projected/b3cb69d8-c6ec-4537-aae4-d202504f2fc8-kube-api-access-hh5lv\") pod \"volume-data-source-validator-549c6d54cf-db627\" (UID: \"b3cb69d8-c6ec-4537-aae4-d202504f2fc8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627"
Jul 24 17:40:27.866006 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:27.865695    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:40:27.866006 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8bnq\" (UniqueName: \"kubernetes.io/projected/b3bc8432-2ec9-46b9-891f-70c53f921b38-kube-api-access-g8bnq\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:27.866006 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:27.865764    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls podName:b3bc8432-2ec9-46b9-891f-70c53f921b38 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:28.365745812 +0000 UTC m=+35.570437591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls") pod "cluster-samples-operator-6748489656-8xjrb" (UID: "b3bc8432-2ec9-46b9-891f-70c53f921b38") : secret "samples-operator-tls" not found
Jul 24 17:40:27.866006 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xns9v\" (UniqueName: \"kubernetes.io/projected/77ceb7e0-86cd-4ee2-81e5-f603c2b8eb44-kube-api-access-xns9v\") pod \"network-check-source-7d88cb55d8-ckhct\" (UID: \"77ceb7e0-86cd-4ee2-81e5-f603c2b8eb44\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct"
Jul 24 17:40:27.866006 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.865873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.866323 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:27.866016    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:27.866323 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:27.866059    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:28.366044935 +0000 UTC m=+35.570736716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : secret "router-metrics-certs-default" not found
Jul 24 17:40:27.866323 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.866052    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/556df0ae-3850-45c5-a889-d36921adf671-config\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.866539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.866476    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/556df0ae-3850-45c5-a889-d36921adf671-trusted-ca\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.868445 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.868423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/556df0ae-3850-45c5-a889-d36921adf671-serving-cert\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.868642 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.868619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-stats-auth\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.868642 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.868634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-default-certificate\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.876767 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.876734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh5lv\" (UniqueName: \"kubernetes.io/projected/b3cb69d8-c6ec-4537-aae4-d202504f2fc8-kube-api-access-hh5lv\") pod \"volume-data-source-validator-549c6d54cf-db627\" (UID: \"b3cb69d8-c6ec-4537-aae4-d202504f2fc8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627"
Jul 24 17:40:27.877162 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.877140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8bnq\" (UniqueName: \"kubernetes.io/projected/b3bc8432-2ec9-46b9-891f-70c53f921b38-kube-api-access-g8bnq\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:27.877371 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.877344    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2hsk\" (UniqueName: \"kubernetes.io/projected/2f5568f4-e4a5-4007-adb7-780b43c081ad-kube-api-access-v2hsk\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:27.877436 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.877393    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsmlr\" (UniqueName: \"kubernetes.io/projected/556df0ae-3850-45c5-a889-d36921adf671-kube-api-access-wsmlr\") pod \"console-operator-584dcb6b54-8gfzp\" (UID: \"556df0ae-3850-45c5-a889-d36921adf671\") " pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.882215 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.882195    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-m7799"
Jul 24 17:40:27.920418 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.920393    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627"
Jul 24 17:40:27.954991 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.954972    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:27.966405 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.966380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hnxf\" (UniqueName: \"kubernetes.io/projected/8dd06a5f-d5af-4e05-bd87-afdbf92c7b43-kube-api-access-8hnxf\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q\" (UID: \"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.966757 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.966566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xns9v\" (UniqueName: \"kubernetes.io/projected/77ceb7e0-86cd-4ee2-81e5-f603c2b8eb44-kube-api-access-xns9v\") pod \"network-check-source-7d88cb55d8-ckhct\" (UID: \"77ceb7e0-86cd-4ee2-81e5-f603c2b8eb44\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct"
Jul 24 17:40:27.966757 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.966629    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8dd06a5f-d5af-4e05-bd87-afdbf92c7b43-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q\" (UID: \"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.966757 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.966656    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8dd06a5f-d5af-4e05-bd87-afdbf92c7b43-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q\" (UID: \"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.967301 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.967280    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8dd06a5f-d5af-4e05-bd87-afdbf92c7b43-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q\" (UID: \"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.968825 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.968805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8dd06a5f-d5af-4e05-bd87-afdbf92c7b43-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q\" (UID: \"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.974175 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.974150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hnxf\" (UniqueName: \"kubernetes.io/projected/8dd06a5f-d5af-4e05-bd87-afdbf92c7b43-kube-api-access-8hnxf\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q\" (UID: \"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:27.974490 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:27.974469    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xns9v\" (UniqueName: \"kubernetes.io/projected/77ceb7e0-86cd-4ee2-81e5-f603c2b8eb44-kube-api-access-xns9v\") pod \"network-check-source-7d88cb55d8-ckhct\" (UID: \"77ceb7e0-86cd-4ee2-81e5-f603c2b8eb44\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct"
Jul 24 17:40:28.027251 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.027206    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"
Jul 24 17:40:28.037348 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.037330    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct"
Jul 24 17:40:28.269195 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.269165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:28.269351 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:28.269331    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:28.269414 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:28.269403    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls podName:a9411acd-1297-42d0-8b2e-bc84a5cfec3b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:29.269386851 +0000 UTC m=+36.474078629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kzcs4" (UID: "a9411acd-1297-42d0-8b2e-bc84a5cfec3b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:28.370416 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.370384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:28.370563 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.370481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:28.370563 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.370508    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:28.370563 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:28.370538    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:28.370714 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:28.370603    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:29.370586797 +0000 UTC m=+36.575278592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : secret "router-metrics-certs-default" not found
Jul 24 17:40:28.370714 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:28.370625    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:29.370608527 +0000 UTC m=+36.575300302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:28.370714 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:28.370636    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:40:28.370714 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:28.370689    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls podName:b3bc8432-2ec9-46b9-891f-70c53f921b38 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:29.370673183 +0000 UTC m=+36.575364973 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls") pod "cluster-samples-operator-6748489656-8xjrb" (UID: "b3bc8432-2ec9-46b9-891f-70c53f921b38") : secret "samples-operator-tls" not found
Jul 24 17:40:28.575765 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.575732    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-m7799"]
Jul 24 17:40:28.759723 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:28.759668    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-cbc40f4e4c0661db81ad2b706b00884027434db4eccb22b67ad1aea2f5c54666 WatchSource:0}: Error finding container cbc40f4e4c0661db81ad2b706b00884027434db4eccb22b67ad1aea2f5c54666: Status 404 returned error can't find the container with id cbc40f4e4c0661db81ad2b706b00884027434db4eccb22b67ad1aea2f5c54666
Jul 24 17:40:28.804207 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.804181    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2"]
Jul 24 17:40:28.807254 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:28.807226    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65571cc3_f16a_4cfa_a73c_50380be899c2.slice/crio-b74ee5b6c1fde990e55b848e0777cfb7f93feda372c4eaea9e8dcf3ffccc073f WatchSource:0}: Error finding container b74ee5b6c1fde990e55b848e0777cfb7f93feda372c4eaea9e8dcf3ffccc073f: Status 404 returned error can't find the container with id b74ee5b6c1fde990e55b848e0777cfb7f93feda372c4eaea9e8dcf3ffccc073f
Jul 24 17:40:28.810068 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.810044    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627"]
Jul 24 17:40:28.813097 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:28.813071    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3cb69d8_c6ec_4537_aae4_d202504f2fc8.slice/crio-ad279b133f9aea62068ab779dafcbbfea4639a39d44f08575182ac50c9088fac WatchSource:0}: Error finding container ad279b133f9aea62068ab779dafcbbfea4639a39d44f08575182ac50c9088fac: Status 404 returned error can't find the container with id ad279b133f9aea62068ab779dafcbbfea4639a39d44f08575182ac50c9088fac
Jul 24 17:40:28.820634 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.820527    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q"]
Jul 24 17:40:28.821580 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.821556    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-8gfzp"]
Jul 24 17:40:28.822357 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:28.822337    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct"]
Jul 24 17:40:28.822873 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:28.822845    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd06a5f_d5af_4e05_bd87_afdbf92c7b43.slice/crio-8db67f9491cf9fe56e4da0edd783275ad3895d0b66365b5587935ff4c0ae3970 WatchSource:0}: Error finding container 8db67f9491cf9fe56e4da0edd783275ad3895d0b66365b5587935ff4c0ae3970: Status 404 returned error can't find the container with id 8db67f9491cf9fe56e4da0edd783275ad3895d0b66365b5587935ff4c0ae3970
Jul 24 17:40:28.824716 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:28.824534    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod556df0ae_3850_45c5_a889_d36921adf671.slice/crio-218e9f3a920aebaf60b303ad881965c975d1c1814f73cc4737eff3a95ffb0575 WatchSource:0}: Error finding container 218e9f3a920aebaf60b303ad881965c975d1c1814f73cc4737eff3a95ffb0575: Status 404 returned error can't find the container with id 218e9f3a920aebaf60b303ad881965c975d1c1814f73cc4737eff3a95ffb0575
Jul 24 17:40:28.825118 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:28.825107    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ceb7e0_86cd_4ee2_81e5_f603c2b8eb44.slice/crio-842a6d3520ec3bc65de71fe10005319451d830ed6e9e9219d0365fa61aaa536e WatchSource:0}: Error finding container 842a6d3520ec3bc65de71fe10005319451d830ed6e9e9219d0365fa61aaa536e: Status 404 returned error can't find the container with id 842a6d3520ec3bc65de71fe10005319451d830ed6e9e9219d0365fa61aaa536e
Jul 24 17:40:29.268834 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.268761    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"]
Jul 24 17:40:29.280023 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.279978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:29.280153 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.280134    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:29.280214 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.280206    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls podName:a9411acd-1297-42d0-8b2e-bc84a5cfec3b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:31.2801865 +0000 UTC m=+38.484878274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kzcs4" (UID: "a9411acd-1297-42d0-8b2e-bc84a5cfec3b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:29.284179 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.284159    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"]
Jul 24 17:40:29.284301 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.284263    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:29.286855 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.286795    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 24 17:40:29.287053 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.287036    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4cj6t\""
Jul 24 17:40:29.287949 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.287322    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 24 17:40:29.381731 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.380791    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:29.381731 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.380839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:29.381731 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.380887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/03625b16-050a-4640-8f1b-b8071d53cdc8-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:29.381731 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.380948    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:29.381731 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.380995    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:29.381731 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.381179    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:29.381731 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.381243    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:31.381224969 +0000 UTC m=+38.585916747 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : secret "router-metrics-certs-default" not found
Jul 24 17:40:29.381731 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.381638    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:31.381623015 +0000 UTC m=+38.586314789 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:29.381731 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.381708    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:40:29.382252 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.381775    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls podName:b3bc8432-2ec9-46b9-891f-70c53f921b38 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:31.381733388 +0000 UTC m=+38.586425163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls") pod "cluster-samples-operator-6748489656-8xjrb" (UID: "b3bc8432-2ec9-46b9-891f-70c53f921b38") : secret "samples-operator-tls" not found
Jul 24 17:40:29.481735 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.481701    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/03625b16-050a-4640-8f1b-b8071d53cdc8-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:29.481918 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.481775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:29.482033 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.482012    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:40:29.482100 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.482089    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert podName:03625b16-050a-4640-8f1b-b8071d53cdc8 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:29.982070643 +0000 UTC m=+37.186762421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-s8skq" (UID: "03625b16-050a-4640-8f1b-b8071d53cdc8") : secret "networking-console-plugin-cert" not found
Jul 24 17:40:29.482969 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.482946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/03625b16-050a-4640-8f1b-b8071d53cdc8-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:29.768799 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.768759    2562 generic.go:356] "Generic (PLEG): container finished" podID="c4707a45-429b-4d07-a7b9-41d3bcf570fa" containerID="c7685d9304d67f487fe3bbf4fe0d7110ef00c0e05c7e4f8c23b2a7143e6acb7d" exitCode=0
Jul 24 17:40:29.768987 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.768850    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tzghr" event={"ID":"c4707a45-429b-4d07-a7b9-41d3bcf570fa","Type":"ContainerDied","Data":"c7685d9304d67f487fe3bbf4fe0d7110ef00c0e05c7e4f8c23b2a7143e6acb7d"}
Jul 24 17:40:29.773405 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.773171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627" event={"ID":"b3cb69d8-c6ec-4537-aae4-d202504f2fc8","Type":"ContainerStarted","Data":"ad279b133f9aea62068ab779dafcbbfea4639a39d44f08575182ac50c9088fac"}
Jul 24 17:40:29.777500 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.777391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2" event={"ID":"65571cc3-f16a-4cfa-a73c-50380be899c2","Type":"ContainerStarted","Data":"b74ee5b6c1fde990e55b848e0777cfb7f93feda372c4eaea9e8dcf3ffccc073f"}
Jul 24 17:40:29.782616 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.782555    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct" event={"ID":"77ceb7e0-86cd-4ee2-81e5-f603c2b8eb44","Type":"ContainerStarted","Data":"842a6d3520ec3bc65de71fe10005319451d830ed6e9e9219d0365fa61aaa536e"}
Jul 24 17:40:29.785684 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.785624    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" event={"ID":"556df0ae-3850-45c5-a889-d36921adf671","Type":"ContainerStarted","Data":"218e9f3a920aebaf60b303ad881965c975d1c1814f73cc4737eff3a95ffb0575"}
Jul 24 17:40:29.788619 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.788562    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m7799" event={"ID":"d66bc6dd-82ce-473f-8aba-de7d3929b6bb","Type":"ContainerStarted","Data":"cbc40f4e4c0661db81ad2b706b00884027434db4eccb22b67ad1aea2f5c54666"}
Jul 24 17:40:29.792489 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.792427    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q" event={"ID":"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43","Type":"ContainerStarted","Data":"8db67f9491cf9fe56e4da0edd783275ad3895d0b66365b5587935ff4c0ae3970"}
Jul 24 17:40:29.987711 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:29.987677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:29.987956 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.987938    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:40:29.988045 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:29.988006    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert podName:03625b16-050a-4640-8f1b-b8071d53cdc8 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:30.987987862 +0000 UTC m=+38.192679643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-s8skq" (UID: "03625b16-050a-4640-8f1b-b8071d53cdc8") : secret "networking-console-plugin-cert" not found
Jul 24 17:40:30.805905 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:30.805825    2562 generic.go:356] "Generic (PLEG): container finished" podID="c4707a45-429b-4d07-a7b9-41d3bcf570fa" containerID="7dce64c38a5d7f12d5f92d1390c254a453d8021c7a61f8986b1c3af198c37671" exitCode=0
Jul 24 17:40:30.807590 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:30.807342    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tzghr" event={"ID":"c4707a45-429b-4d07-a7b9-41d3bcf570fa","Type":"ContainerDied","Data":"7dce64c38a5d7f12d5f92d1390c254a453d8021c7a61f8986b1c3af198c37671"}
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:30.996817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:30.996909    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:30.996953    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:30.997755    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:30.997814    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls podName:d93ed42c-d048-484d-941a-93193f1e13cd nodeName:}" failed. No retries permitted until 2026-07-24 17:40:38.9977938 +0000 UTC m=+46.202485581 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls") pod "dns-default-7jczg" (UID: "d93ed42c-d048-484d-941a-93193f1e13cd") : secret "dns-default-metrics-tls" not found
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:30.997975    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:30.998018    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert podName:03625b16-050a-4640-8f1b-b8071d53cdc8 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:32.998004537 +0000 UTC m=+40.202696319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-s8skq" (UID: "03625b16-050a-4640-8f1b-b8071d53cdc8") : secret "networking-console-plugin-cert" not found
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:30.998381    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:30.998396    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d8dbcb57-tglbk: secret "image-registry-tls" not found
Jul 24 17:40:30.998508 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:30.998436    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls podName:e6011f48-0283-430a-8d6a-e81ce31eec5b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:38.998422124 +0000 UTC m=+46.203113945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls") pod "image-registry-55d8dbcb57-tglbk" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b") : secret "image-registry-tls" not found
Jul 24 17:40:31.299753 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:31.299228    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:31.299753 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:31.299362    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:31.299753 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:31.299424    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls podName:a9411acd-1297-42d0-8b2e-bc84a5cfec3b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:35.299405637 +0000 UTC m=+42.504097416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kzcs4" (UID: "a9411acd-1297-42d0-8b2e-bc84a5cfec3b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:31.400300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:31.400232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:31.400300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:31.400275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:31.400503 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:31.400366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:31.400503 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:31.400383    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:35.40036055 +0000 UTC m=+42.605052342 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:31.400503 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:31.400444    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:40:31.400701 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:31.400506    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls podName:b3bc8432-2ec9-46b9-891f-70c53f921b38 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:35.400488855 +0000 UTC m=+42.605180638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls") pod "cluster-samples-operator-6748489656-8xjrb" (UID: "b3bc8432-2ec9-46b9-891f-70c53f921b38") : secret "samples-operator-tls" not found
Jul 24 17:40:31.400701 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:31.400535    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:31.400701 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:31.400572    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:35.400560692 +0000 UTC m=+42.605252470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : secret "router-metrics-certs-default" not found
Jul 24 17:40:31.816269 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:31.816235    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tzghr" event={"ID":"c4707a45-429b-4d07-a7b9-41d3bcf570fa","Type":"ContainerStarted","Data":"f2ec3521bb0c66cbd773841867d369290429843a49ba537031ab687387b82d8b"}
Jul 24 17:40:31.841255 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:31.840845    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tzghr" podStartSLOduration=6.968854902 podStartE2EDuration="37.840823983s" totalImagesPullingTime="30.871969081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:31.840133519 +0000 UTC m=+39.044825350" watchObservedRunningTime="2026-07-24 17:40:31.840823983 +0000 UTC m=+39.045515781"
Jul 24 17:40:33.015484 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:33.015449    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:33.015983 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:33.015611    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:40:33.015983 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:33.015692    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert podName:03625b16-050a-4640-8f1b-b8071d53cdc8 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:37.015670394 +0000 UTC m=+44.220362183 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-s8skq" (UID: "03625b16-050a-4640-8f1b-b8071d53cdc8") : secret "networking-console-plugin-cert" not found
Jul 24 17:40:35.334078 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:35.334040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:35.334528 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:35.334196    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:35.334528 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:35.334263    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls podName:a9411acd-1297-42d0-8b2e-bc84a5cfec3b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:43.334244586 +0000 UTC m=+50.538936366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kzcs4" (UID: "a9411acd-1297-42d0-8b2e-bc84a5cfec3b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:35.435027 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:35.435002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:35.435027 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:35.435029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:35.435213 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:35.435086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:35.435213 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:35.435163    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:43.435144899 +0000 UTC m=+50.639836690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:35.435320 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:35.435223    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:40:35.435320 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:35.435234    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:35.435320 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:35.435267    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls podName:b3bc8432-2ec9-46b9-891f-70c53f921b38 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:43.435255213 +0000 UTC m=+50.639947001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls") pod "cluster-samples-operator-6748489656-8xjrb" (UID: "b3bc8432-2ec9-46b9-891f-70c53f921b38") : secret "samples-operator-tls" not found
Jul 24 17:40:35.435320 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:35.435280    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:43.435275116 +0000 UTC m=+50.639966890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : secret "router-metrics-certs-default" not found
Jul 24 17:40:37.046962 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.046931    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:37.047332 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:37.047084    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:40:37.047332 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:37.047150    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert podName:03625b16-050a-4640-8f1b-b8071d53cdc8 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:45.047134101 +0000 UTC m=+52.251825875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-s8skq" (UID: "03625b16-050a-4640-8f1b-b8071d53cdc8") : secret "networking-console-plugin-cert" not found
Jul 24 17:40:37.838841 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.838101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" event={"ID":"556df0ae-3850-45c5-a889-d36921adf671","Type":"ContainerStarted","Data":"e915046a1ca9a981dc83c34781de894d9c97b4123292e0fca0c2768baf024a1e"}
Jul 24 17:40:37.838841 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.838621    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:37.840492 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.839892    2562 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-8gfzp container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.15:8443/readyz\": dial tcp 10.132.0.15:8443: connect: connection refused" start-of-body=
Jul 24 17:40:37.840492 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.839949    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.15:8443/readyz\": dial tcp 10.132.0.15:8443: connect: connection refused"
Jul 24 17:40:37.843297 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.842810    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m7799" event={"ID":"d66bc6dd-82ce-473f-8aba-de7d3929b6bb","Type":"ContainerStarted","Data":"21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db"}
Jul 24 17:40:37.845540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.845209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q" event={"ID":"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43","Type":"ContainerStarted","Data":"5c7f1439b647acc12d6f46dca07499f34951b900542e5cebd3c9bc15306d0e55"}
Jul 24 17:40:37.847092 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.847044    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mf7bb" event={"ID":"0ad9ca8c-7fdf-4e9e-bb7f-4bd3c962127b","Type":"ContainerStarted","Data":"f7c05e2907dbf699fc00ce279d0849aa9d5df505bf1a7f2145680165d9b31c8c"}
Jul 24 17:40:37.847092 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.847081    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:40:37.848600 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.848559    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627" event={"ID":"b3cb69d8-c6ec-4537-aae4-d202504f2fc8","Type":"ContainerStarted","Data":"0b604e60d7d3d39950d743e0d306103b26598928e0a9681b283f3f791a435b6f"}
Jul 24 17:40:37.850014 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.849971    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2" event={"ID":"65571cc3-f16a-4cfa-a73c-50380be899c2","Type":"ContainerStarted","Data":"1f57a71306fc3d131a55f4fa3ea2cb3c9a787b22a96bd8ddd01923861a9197d5"}
Jul 24 17:40:37.851416 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.851398    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct" event={"ID":"77ceb7e0-86cd-4ee2-81e5-f603c2b8eb44","Type":"ContainerStarted","Data":"a78b985dbf4424876fced3e4cbfe8b271c0e48aa39d8a7ea89eabbac77df2b66"}
Jul 24 17:40:37.855445 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.855402    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podStartSLOduration=2.126533777 podStartE2EDuration="10.85538877s" totalImagesPullingTime="8.728854993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:37.853502374 +0000 UTC m=+45.058194171" watchObservedRunningTime="2026-07-24 17:40:37.85538877 +0000 UTC m=+45.060080568"
Jul 24 17:40:37.868241 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.867840    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mf7bb" podStartSLOduration=33.0232593 podStartE2EDuration="43.867828162s" totalImagesPullingTime="10.844568862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:37.867498906 +0000 UTC m=+45.072190704" watchObservedRunningTime="2026-07-24 17:40:37.867828162 +0000 UTC m=+45.072519960"
Jul 24 17:40:37.882607 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.882552    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2" podStartSLOduration=2.137004661 podStartE2EDuration="10.882540421s" totalImagesPullingTime="8.74553576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:37.881629896 +0000 UTC m=+45.086321694" watchObservedRunningTime="2026-07-24 17:40:37.882540421 +0000 UTC m=+45.087232218"
Jul 24 17:40:37.895964 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.895580    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q" podStartSLOduration=2.165306912 podStartE2EDuration="10.895563368s" totalImagesPullingTime="8.730256456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:37.894671338 +0000 UTC m=+45.099363137" watchObservedRunningTime="2026-07-24 17:40:37.895563368 +0000 UTC m=+45.100255166"
Jul 24 17:40:37.920428 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.920073    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-ckhct" podStartSLOduration=2.192590635 podStartE2EDuration="10.920057153s" totalImagesPullingTime="8.727466518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:37.91895594 +0000 UTC m=+45.123647739" watchObservedRunningTime="2026-07-24 17:40:37.920057153 +0000 UTC m=+45.124748950"
Jul 24 17:40:37.933257 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.932382    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-db627" podStartSLOduration=2.33570852 podStartE2EDuration="10.932374113s" totalImagesPullingTime="8.596665593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:37.93211304 +0000 UTC m=+45.136804839" watchObservedRunningTime="2026-07-24 17:40:37.932374113 +0000 UTC m=+45.137065900"
Jul 24 17:40:37.958464 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.958439    2562 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-8gfzp container/console-operator namespace/openshift-console-operator: Liveness probe status=failure output="Get \"https://10.132.0.15:8443/healthz\": dial tcp 10.132.0.15:8443: connect: connection refused" start-of-body=
Jul 24 17:40:37.958652 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:37.958626    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.15:8443/healthz\": dial tcp 10.132.0.15:8443: connect: connection refused"
Jul 24 17:40:38.757321 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.757262    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-m7799" podStartSLOduration=2.984875109 podStartE2EDuration="11.757241614s" totalImagesPullingTime="8.772366505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:37.948107533 +0000 UTC m=+45.152799331" watchObservedRunningTime="2026-07-24 17:40:38.757241614 +0000 UTC m=+45.961933413"
Jul 24 17:40:38.757787 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.757485    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xcppd"]
Jul 24 17:40:38.770198 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.770163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.771540 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.771510    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xcppd"]
Jul 24 17:40:38.772300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.772277    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g9gf7\""
Jul 24 17:40:38.772448 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.772428    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 17:40:38.772578 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.772389    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 17:40:38.856472 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.856443    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/0.log"
Jul 24 17:40:38.856639 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.856485    2562 generic.go:356] "Generic (PLEG): container finished" podID="556df0ae-3850-45c5-a889-d36921adf671" containerID="e915046a1ca9a981dc83c34781de894d9c97b4123292e0fca0c2768baf024a1e" exitCode=255
Jul 24 17:40:38.856639 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.856573    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" event={"ID":"556df0ae-3850-45c5-a889-d36921adf671","Type":"ContainerDied","Data":"e915046a1ca9a981dc83c34781de894d9c97b4123292e0fca0c2768baf024a1e"}
Jul 24 17:40:38.856909 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.856863    2562 scope.go:117] "RemoveContainer" containerID="e915046a1ca9a981dc83c34781de894d9c97b4123292e0fca0c2768baf024a1e"
Jul 24 17:40:38.858408 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.858383    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8fsnd" event={"ID":"74b9bf45-8bba-459e-bd93-ee92f990ae95","Type":"ContainerStarted","Data":"6f4ef53fed75df3adf8bd109cb524c39c99d2a9f9a8d9b31a22ce5245a545647"}
Jul 24 17:40:38.867237 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.867215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.867320 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.867262    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ae081de-f739-45a4-a66e-7f9d9e56f35b-crio-socket\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.867368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.867323    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dblcq\" (UniqueName: \"kubernetes.io/projected/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-api-access-dblcq\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.867368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.867353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.867439 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.867379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ae081de-f739-45a4-a66e-7f9d9e56f35b-data-volume\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.886776 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.885707    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8fsnd" podStartSLOduration=34.032068472 podStartE2EDuration="44.885691958s" totalImagesPullingTime="10.853623486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:38.885619598 +0000 UTC m=+46.090311396" watchObservedRunningTime="2026-07-24 17:40:38.885691958 +0000 UTC m=+46.090383753"
Jul 24 17:40:38.968701 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.968670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.968812 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:38.968779    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:38.968902 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:38.968830    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:39.468812708 +0000 UTC m=+46.673504489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:38.968981 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.968904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ae081de-f739-45a4-a66e-7f9d9e56f35b-crio-socket\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.969048 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.969017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dblcq\" (UniqueName: \"kubernetes.io/projected/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-api-access-dblcq\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.969132 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.969108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.969197 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.969129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ae081de-f739-45a4-a66e-7f9d9e56f35b-crio-socket\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.969245 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.969197    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ae081de-f739-45a4-a66e-7f9d9e56f35b-data-volume\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.969299 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:38.969257    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:38.969355 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:38.969305    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:39.469288021 +0000 UTC m=+46.673979808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:38.969549 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.969524    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ae081de-f739-45a4-a66e-7f9d9e56f35b-data-volume\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:38.979230 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:38.979209    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dblcq\" (UniqueName: \"kubernetes.io/projected/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-api-access-dblcq\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:39.070076 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.070055    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:39.070184 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.070105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:39.070234 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.070194    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:40:39.070234 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.070203    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d8dbcb57-tglbk: secret "image-registry-tls" not found
Jul 24 17:40:39.070234 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.070209    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:40:39.070341 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.070236    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls podName:e6011f48-0283-430a-8d6a-e81ce31eec5b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:55.070225272 +0000 UTC m=+62.274917050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls") pod "image-registry-55d8dbcb57-tglbk" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b") : secret "image-registry-tls" not found
Jul 24 17:40:39.070341 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.070264    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls podName:d93ed42c-d048-484d-941a-93193f1e13cd nodeName:}" failed. No retries permitted until 2026-07-24 17:40:55.07024742 +0000 UTC m=+62.274939200 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls") pod "dns-default-7jczg" (UID: "d93ed42c-d048-484d-941a-93193f1e13cd") : secret "dns-default-metrics-tls" not found
Jul 24 17:40:39.473074 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.473010    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:39.473074 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.473070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:39.473214 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.473143    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:39.473214 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.473167    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:39.473214 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.473199    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:40.473184948 +0000 UTC m=+47.677876723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:39.473214 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.473212    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:40.473205955 +0000 UTC m=+47.677897731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:39.863545 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.863521    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/1.log"
Jul 24 17:40:39.863910 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.863861    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/0.log"
Jul 24 17:40:39.863958 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.863909    2562 generic.go:356] "Generic (PLEG): container finished" podID="556df0ae-3850-45c5-a889-d36921adf671" containerID="b346f72edc0ac3f72043a580877db911ff239dc672c6ddc862691c80642f85b5" exitCode=255
Jul 24 17:40:39.864008 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.863991    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" event={"ID":"556df0ae-3850-45c5-a889-d36921adf671","Type":"ContainerDied","Data":"b346f72edc0ac3f72043a580877db911ff239dc672c6ddc862691c80642f85b5"}
Jul 24 17:40:39.864053 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.864040    2562 scope.go:117] "RemoveContainer" containerID="e915046a1ca9a981dc83c34781de894d9c97b4123292e0fca0c2768baf024a1e"
Jul 24 17:40:39.864238 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:39.864221    2562 scope.go:117] "RemoveContainer" containerID="b346f72edc0ac3f72043a580877db911ff239dc672c6ddc862691c80642f85b5"
Jul 24 17:40:39.864402 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:39.864374    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-8gfzp_openshift-console-operator(556df0ae-3850-45c5-a889-d36921adf671)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671"
Jul 24 17:40:40.155966 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:40.155908    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t4rbd_291c612c-f9bd-4525-a4b6-e3f6777def41/dns-node-resolver/0.log"
Jul 24 17:40:40.482097 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:40.482022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:40.482246 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:40.482117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:40.482246 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:40.482126    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:40.482246 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:40.482190    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:42.482176122 +0000 UTC m=+49.686867901 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:40.482246 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:40.482198    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:40.482246 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:40.482244    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:42.482233606 +0000 UTC m=+49.686925381 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:40.868855 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:40.868830    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/1.log"
Jul 24 17:40:40.869222 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:40.869155    2562 scope.go:117] "RemoveContainer" containerID="b346f72edc0ac3f72043a580877db911ff239dc672c6ddc862691c80642f85b5"
Jul 24 17:40:40.869315 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:40.869297    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-8gfzp_openshift-console-operator(556df0ae-3850-45c5-a889-d36921adf671)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671"
Jul 24 17:40:41.151961 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:41.151870    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t4t99_81d447a3-e3df-4850-bcd4-60ef6e80cbcd/node-ca/0.log"
Jul 24 17:40:41.552132 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:41.552113    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v622r_cc2d1617-bb81-4541-9971-09854abe6150/serve-healthcheck-canary/0.log"
Jul 24 17:40:42.496057 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:42.496024    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:42.496476 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:42.496092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:42.496476 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:42.496173    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:42.496476 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:42.496205    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:42.496476 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:42.496245    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:46.496229228 +0000 UTC m=+53.700921002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:42.496476 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:42.496273    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:46.496257372 +0000 UTC m=+53.700949147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:43.403679 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:43.403648    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:43.403832 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:43.403790    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:43.403904 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:43.403864    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls podName:a9411acd-1297-42d0-8b2e-bc84a5cfec3b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:59.403845507 +0000 UTC m=+66.608537284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kzcs4" (UID: "a9411acd-1297-42d0-8b2e-bc84a5cfec3b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 17:40:43.504265 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:43.504239    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:43.504265 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:43.504268    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:43.504624 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:43.504397    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:59.504380825 +0000 UTC m=+66.709072613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:43.504624 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:43.504401    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:40:43.504624 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:43.504429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:43.504624 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:43.504439    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls podName:b3bc8432-2ec9-46b9-891f-70c53f921b38 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:59.504428302 +0000 UTC m=+66.709120092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls") pod "cluster-samples-operator-6748489656-8xjrb" (UID: "b3bc8432-2ec9-46b9-891f-70c53f921b38") : secret "samples-operator-tls" not found
Jul 24 17:40:43.504624 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:43.504486    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:43.504624 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:43.504526    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs podName:2f5568f4-e4a5-4007-adb7-780b43c081ad nodeName:}" failed. No retries permitted until 2026-07-24 17:40:59.504515487 +0000 UTC m=+66.709207262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs") pod "router-default-5cb6869597-2kxmd" (UID: "2f5568f4-e4a5-4007-adb7-780b43c081ad") : secret "router-metrics-certs-default" not found
Jul 24 17:40:45.115672 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:45.115630    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:40:45.116113 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:45.115799    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:40:45.116113 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:45.115867    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert podName:03625b16-050a-4640-8f1b-b8071d53cdc8 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:01.115849718 +0000 UTC m=+68.320541504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-s8skq" (UID: "03625b16-050a-4640-8f1b-b8071d53cdc8") : secret "networking-console-plugin-cert" not found
Jul 24 17:40:46.527031 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:46.527000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:46.527409 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:46.527069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:46.527409 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:46.527179    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:46.527409 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:46.527239    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:54.527225255 +0000 UTC m=+61.731917033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:46.527409 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:46.527179    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:46.527409 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:46.527310    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:40:54.527298433 +0000 UTC m=+61.731990213 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:47.838621 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:47.838587    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:47.839064 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:47.838930    2562 scope.go:117] "RemoveContainer" containerID="b346f72edc0ac3f72043a580877db911ff239dc672c6ddc862691c80642f85b5"
Jul 24 17:40:47.839105 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:47.839074    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-8gfzp_openshift-console-operator(556df0ae-3850-45c5-a889-d36921adf671)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671"
Jul 24 17:40:47.955655 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:47.955628    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:40:47.955972 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:47.955958    2562 scope.go:117] "RemoveContainer" containerID="b346f72edc0ac3f72043a580877db911ff239dc672c6ddc862691c80642f85b5"
Jul 24 17:40:47.956117 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:47.956096    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-8gfzp_openshift-console-operator(556df0ae-3850-45c5-a889-d36921adf671)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671"
Jul 24 17:40:48.731153 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:48.731131    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r59tv"
Jul 24 17:40:54.592629 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:54.592584    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:54.593055 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:54.592693    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:54.593055 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:54.592801    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:54.593055 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:40:54.592853    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls podName:2ae081de-f739-45a4-a66e-7f9d9e56f35b nodeName:}" failed. No retries permitted until 2026-07-24 17:41:10.592837967 +0000 UTC m=+77.797529745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xcppd" (UID: "2ae081de-f739-45a4-a66e-7f9d9e56f35b") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:54.593181 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:54.593138    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae081de-f739-45a4-a66e-7f9d9e56f35b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:40:55.096156 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.096121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:55.096324 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.096182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:55.098779 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.098752    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"image-registry-55d8dbcb57-tglbk\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") " pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:55.098915 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.098818    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d93ed42c-d048-484d-941a-93193f1e13cd-metrics-tls\") pod \"dns-default-7jczg\" (UID: \"d93ed42c-d048-484d-941a-93193f1e13cd\") " pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:55.259576 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.259555    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-crf7b\""
Jul 24 17:40:55.268074 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.268050    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:55.273818 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.273800    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tq4qz\""
Jul 24 17:40:55.281599 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.281581    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:55.452482 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.452451    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55d8dbcb57-tglbk"]
Jul 24 17:40:55.455276 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:55.455250    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6011f48_0283_430a_8d6a_e81ce31eec5b.slice/crio-6dd27b86e50518ad28e58cbabea2881224e9e1d79de67bca52b4e1e16dafc28b WatchSource:0}: Error finding container 6dd27b86e50518ad28e58cbabea2881224e9e1d79de67bca52b4e1e16dafc28b: Status 404 returned error can't find the container with id 6dd27b86e50518ad28e58cbabea2881224e9e1d79de67bca52b4e1e16dafc28b
Jul 24 17:40:55.477207 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.477186    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7jczg"]
Jul 24 17:40:55.479574 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:55.479550    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd93ed42c_d048_484d_941a_93193f1e13cd.slice/crio-70908908910a96f64b584b7b681960e5d04f9f58da83eaf14c54811ff2c3b4ac WatchSource:0}: Error finding container 70908908910a96f64b584b7b681960e5d04f9f58da83eaf14c54811ff2c3b4ac: Status 404 returned error can't find the container with id 70908908910a96f64b584b7b681960e5d04f9f58da83eaf14c54811ff2c3b4ac
Jul 24 17:40:55.921365 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.921331    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk" event={"ID":"e6011f48-0283-430a-8d6a-e81ce31eec5b","Type":"ContainerStarted","Data":"df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637"}
Jul 24 17:40:55.921793 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.921395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk" event={"ID":"e6011f48-0283-430a-8d6a-e81ce31eec5b","Type":"ContainerStarted","Data":"6dd27b86e50518ad28e58cbabea2881224e9e1d79de67bca52b4e1e16dafc28b"}
Jul 24 17:40:55.921793 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.921423    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:40:55.922536 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.922509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7jczg" event={"ID":"d93ed42c-d048-484d-941a-93193f1e13cd","Type":"ContainerStarted","Data":"70908908910a96f64b584b7b681960e5d04f9f58da83eaf14c54811ff2c3b4ac"}
Jul 24 17:40:55.943808 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:55.943770    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk" podStartSLOduration=61.943758131 podStartE2EDuration="1m1.943758131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:55.942518035 +0000 UTC m=+63.147209833" watchObservedRunningTime="2026-07-24 17:40:55.943758131 +0000 UTC m=+63.148449927"
Jul 24 17:40:57.930675 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:57.930649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7jczg" event={"ID":"d93ed42c-d048-484d-941a-93193f1e13cd","Type":"ContainerStarted","Data":"194c5b98c8897307a94c60984b0a98cf19f577ac7d9efbb74c8e4fb243ae586a"}
Jul 24 17:40:58.122670 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:58.122645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:58.124765 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:58.124740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64665c61-db20-4439-a490-af90a1d0e579-metrics-certs\") pod \"network-metrics-daemon-kz48t\" (UID: \"64665c61-db20-4439-a490-af90a1d0e579\") " pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:58.315817 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:58.315792    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-62mvz\""
Jul 24 17:40:58.323837 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:58.323820    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kz48t"
Jul 24 17:40:58.458855 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:58.458829    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kz48t"]
Jul 24 17:40:58.461838 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:58.461812    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64665c61_db20_4439_a490_af90a1d0e579.slice/crio-82073a513bc975f43ab2bd11172577e11436c89d1f4f2680ecb61f0870541f8e WatchSource:0}: Error finding container 82073a513bc975f43ab2bd11172577e11436c89d1f4f2680ecb61f0870541f8e: Status 404 returned error can't find the container with id 82073a513bc975f43ab2bd11172577e11436c89d1f4f2680ecb61f0870541f8e
Jul 24 17:40:58.936730 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:58.936691    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7jczg" event={"ID":"d93ed42c-d048-484d-941a-93193f1e13cd","Type":"ContainerStarted","Data":"d6057b92165cda6af3e06b29514f9c4b75705e5c5a1bdbffea3f92649f64c330"}
Jul 24 17:40:58.937134 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:58.936806    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7jczg"
Jul 24 17:40:58.938023 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:58.937997    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kz48t" event={"ID":"64665c61-db20-4439-a490-af90a1d0e579","Type":"ContainerStarted","Data":"82073a513bc975f43ab2bd11172577e11436c89d1f4f2680ecb61f0870541f8e"}
Jul 24 17:40:58.953418 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:58.953359    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7jczg" podStartSLOduration=33.718848129 podStartE2EDuration="35.953345031s" totalImagesPullingTime="2.234496902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:58.952537464 +0000 UTC m=+66.157229262" watchObservedRunningTime="2026-07-24 17:40:58.953345031 +0000 UTC m=+66.158036828"
Jul 24 17:40:59.435150 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.435116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:59.437526 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.437501    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9411acd-1297-42d0-8b2e-bc84a5cfec3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kzcs4\" (UID: \"a9411acd-1297-42d0-8b2e-bc84a5cfec3b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:59.535600 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.535570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:59.535765 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.535606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:59.535830 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.535799    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:59.536358 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.536331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f5568f4-e4a5-4007-adb7-780b43c081ad-service-ca-bundle\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:59.538084 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.538065    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3bc8432-2ec9-46b9-891f-70c53f921b38-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-8xjrb\" (UID: \"b3bc8432-2ec9-46b9-891f-70c53f921b38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:59.538199 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.538140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f5568f4-e4a5-4007-adb7-780b43c081ad-metrics-certs\") pod \"router-default-5cb6869597-2kxmd\" (UID: \"2f5568f4-e4a5-4007-adb7-780b43c081ad\") " pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:59.667469 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.667443    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wm84m\""
Jul 24 17:40:59.675236 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.675218    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"
Jul 24 17:40:59.702978 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.702959    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c2x66\""
Jul 24 17:40:59.711790 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.711763    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:40:59.742649 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.742623    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wq865\""
Jul 24 17:40:59.751243 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.750442    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"
Jul 24 17:40:59.828510 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.828480    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4"]
Jul 24 17:40:59.832298 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:40:59.832256    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9411acd_1297_42d0_8b2e_bc84a5cfec3b.slice/crio-8e56bfe02aafde2b066b6a2006e94cdca8429f33d4fcfbba54979963fd10ff53 WatchSource:0}: Error finding container 8e56bfe02aafde2b066b6a2006e94cdca8429f33d4fcfbba54979963fd10ff53: Status 404 returned error can't find the container with id 8e56bfe02aafde2b066b6a2006e94cdca8429f33d4fcfbba54979963fd10ff53
Jul 24 17:40:59.878148 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.878119    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5cb6869597-2kxmd"]
Jul 24 17:40:59.919628 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.919604    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb"]
Jul 24 17:40:59.942063 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:40:59.942032    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4" event={"ID":"a9411acd-1297-42d0-8b2e-bc84a5cfec3b","Type":"ContainerStarted","Data":"8e56bfe02aafde2b066b6a2006e94cdca8429f33d4fcfbba54979963fd10ff53"}
Jul 24 17:41:00.045422 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:41:00.045397    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5568f4_e4a5_4007_adb7_780b43c081ad.slice/crio-8f54e34e7aaf83f5f2a3f866fdecfcc5bbf78b73d75e389cd8d761d101701a8c WatchSource:0}: Error finding container 8f54e34e7aaf83f5f2a3f866fdecfcc5bbf78b73d75e389cd8d761d101701a8c: Status 404 returned error can't find the container with id 8f54e34e7aaf83f5f2a3f866fdecfcc5bbf78b73d75e389cd8d761d101701a8c
Jul 24 17:41:00.949313 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:00.949278    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5cb6869597-2kxmd" event={"ID":"2f5568f4-e4a5-4007-adb7-780b43c081ad","Type":"ContainerStarted","Data":"e9e5f4ac024a4a15932b25d974a9bdceb9b821b7a2e5be5cea0118e39e710de5"}
Jul 24 17:41:00.949779 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:00.949384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5cb6869597-2kxmd" event={"ID":"2f5568f4-e4a5-4007-adb7-780b43c081ad","Type":"ContainerStarted","Data":"8f54e34e7aaf83f5f2a3f866fdecfcc5bbf78b73d75e389cd8d761d101701a8c"}
Jul 24 17:41:00.951494 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:00.951415    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kz48t" event={"ID":"64665c61-db20-4439-a490-af90a1d0e579","Type":"ContainerStarted","Data":"bdd843f8e4a0c0ee3548c66cd3f7c7a9950ac88821f9aa59b7db6ab2a91a9c1b"}
Jul 24 17:41:00.951494 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:00.951470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kz48t" event={"ID":"64665c61-db20-4439-a490-af90a1d0e579","Type":"ContainerStarted","Data":"e0c6d745278182ff6a0b73aa267c7b6aab986bfd367e7f9e729d803e4bf45e18"}
Jul 24 17:41:00.952756 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:00.952729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb" event={"ID":"b3bc8432-2ec9-46b9-891f-70c53f921b38","Type":"ContainerStarted","Data":"4b4b8bdc4325f77940cc4ea1e6a173ab0a90af6322a8c726713bd2981acf19cf"}
Jul 24 17:41:00.971266 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:00.971225    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5cb6869597-2kxmd" podStartSLOduration=33.971210806 podStartE2EDuration="33.971210806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:00.96953993 +0000 UTC m=+68.174231728" watchObservedRunningTime="2026-07-24 17:41:00.971210806 +0000 UTC m=+68.175902606"
Jul 24 17:41:00.988836 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:00.988788    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kz48t" podStartSLOduration=65.376859057 podStartE2EDuration="1m6.988770052s" totalImagesPullingTime="1.611910995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:00.987269462 +0000 UTC m=+68.191961262" watchObservedRunningTime="2026-07-24 17:41:00.988770052 +0000 UTC m=+68.193461850"
Jul 24 17:41:01.148506 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:01.148474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:41:01.150799 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:01.150779    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03625b16-050a-4640-8f1b-b8071d53cdc8-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-s8skq\" (UID: \"03625b16-050a-4640-8f1b-b8071d53cdc8\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:41:01.402362 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:01.402329    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4cj6t\""
Jul 24 17:41:01.410285 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:01.410262    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"
Jul 24 17:41:01.712062 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:01.711987    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:41:01.714890 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:01.714854    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:41:01.957168 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:01.957118    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:41:01.958275 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:01.958248    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5cb6869597-2kxmd"
Jul 24 17:41:02.239734 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.239712    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-s8skq"]
Jul 24 17:41:02.242514 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:41:02.242470    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03625b16_050a_4640_8f1b_b8071d53cdc8.slice/crio-6caea47d5f2fc271a5a3b2035e6ee3ffea000350c6363eb87e34a3f3efe82504 WatchSource:0}: Error finding container 6caea47d5f2fc271a5a3b2035e6ee3ffea000350c6363eb87e34a3f3efe82504: Status 404 returned error can't find the container with id 6caea47d5f2fc271a5a3b2035e6ee3ffea000350c6363eb87e34a3f3efe82504
Jul 24 17:41:02.402368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.402295    2562 scope.go:117] "RemoveContainer" containerID="b346f72edc0ac3f72043a580877db911ff239dc672c6ddc862691c80642f85b5"
Jul 24 17:41:02.962322 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.962285    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb" event={"ID":"b3bc8432-2ec9-46b9-891f-70c53f921b38","Type":"ContainerStarted","Data":"4961c32fff8107a521cc3e22061438ef9468098b417b44ac825e83d04639588f"}
Jul 24 17:41:02.962735 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.962347    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb" event={"ID":"b3bc8432-2ec9-46b9-891f-70c53f921b38","Type":"ContainerStarted","Data":"c6809312dce4ad79c2a5983b30b091ef7b2332005537bfa8680dc43c84497bdb"}
Jul 24 17:41:02.964207 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.964186    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:41:02.964634 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.964615    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/1.log"
Jul 24 17:41:02.964766 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.964651    2562 generic.go:356] "Generic (PLEG): container finished" podID="556df0ae-3850-45c5-a889-d36921adf671" containerID="00759e559141f4ffe5c56f256148abcdc26fdf42420720dee84d3cb5b4d246f7" exitCode=255
Jul 24 17:41:02.964766 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.964687    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" event={"ID":"556df0ae-3850-45c5-a889-d36921adf671","Type":"ContainerDied","Data":"00759e559141f4ffe5c56f256148abcdc26fdf42420720dee84d3cb5b4d246f7"}
Jul 24 17:41:02.964766 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.964736    2562 scope.go:117] "RemoveContainer" containerID="b346f72edc0ac3f72043a580877db911ff239dc672c6ddc862691c80642f85b5"
Jul 24 17:41:02.965725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.965012    2562 scope.go:117] "RemoveContainer" containerID="00759e559141f4ffe5c56f256148abcdc26fdf42420720dee84d3cb5b4d246f7"
Jul 24 17:41:02.965725 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:02.965180    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-8gfzp_openshift-console-operator(556df0ae-3850-45c5-a889-d36921adf671)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671"
Jul 24 17:41:02.966730 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.966697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4" event={"ID":"a9411acd-1297-42d0-8b2e-bc84a5cfec3b","Type":"ContainerStarted","Data":"515d74f2129b4755a8b307c33ef6b4ac090441862e08c5fd3a8dd8745ec8deda"}
Jul 24 17:41:02.968526 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.968504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq" event={"ID":"03625b16-050a-4640-8f1b-b8071d53cdc8","Type":"ContainerStarted","Data":"6caea47d5f2fc271a5a3b2035e6ee3ffea000350c6363eb87e34a3f3efe82504"}
Jul 24 17:41:02.978721 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:02.978683    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-8xjrb" podStartSLOduration=33.973939161 podStartE2EDuration="35.978671801s" totalImagesPullingTime="2.00473264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:02.977428353 +0000 UTC m=+70.182120150" watchObservedRunningTime="2026-07-24 17:41:02.978671801 +0000 UTC m=+70.183363597"
Jul 24 17:41:03.006284 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.006238    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kzcs4" podStartSLOduration=33.745990175 podStartE2EDuration="36.006220064s" totalImagesPullingTime="2.260229889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:03.005497088 +0000 UTC m=+70.210188884" watchObservedRunningTime="2026-07-24 17:41:03.006220064 +0000 UTC m=+70.210911863"
Jul 24 17:41:03.608581 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.608547    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"]
Jul 24 17:41:03.610727 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.610709    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"
Jul 24 17:41:03.613448 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.613427    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sm7k6\""
Jul 24 17:41:03.613448 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.613438    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 17:41:03.614055 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.614036    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 17:41:03.614159 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.614064    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 24 17:41:03.614224 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.614212    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 17:41:03.629361 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.629343    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"]
Jul 24 17:41:03.718098 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.718053    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"]
Jul 24 17:41:03.720032 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.720015    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"
Jul 24 17:41:03.722409 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:03.722379    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"prometheus-operator-admission-webhook-7f8d78f54b-qgzlf\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="c421f0aa-be5d-4fa9-8b3a-40db518783a3" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"
Jul 24 17:41:03.722490 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:03.722426    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-admission-webhook-tls\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" type="*v1.Secret"
Jul 24 17:41:03.731694 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:03.731669    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-admission-webhook-dockercfg-pkshd\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-pkshd\"" type="*v1.Secret"
Jul 24 17:41:03.741284 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.741265    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"]
Jul 24 17:41:03.765612 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.765585    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bv68\" (UniqueName: \"kubernetes.io/projected/40244b57-1099-4853-be6e-ef487a3abdd4-kube-api-access-8bv68\") pod \"managed-serviceaccount-addon-agent-ff97ccffd-zwrlr\" (UID: \"40244b57-1099-4853-be6e-ef487a3abdd4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"
Jul 24 17:41:03.765612 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.765613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40244b57-1099-4853-be6e-ef487a3abdd4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-ff97ccffd-zwrlr\" (UID: \"40244b57-1099-4853-be6e-ef487a3abdd4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"
Jul 24 17:41:03.866868 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.866843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c421f0aa-be5d-4fa9-8b3a-40db518783a3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qgzlf\" (UID: \"c421f0aa-be5d-4fa9-8b3a-40db518783a3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"
Jul 24 17:41:03.866976 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.866892    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bv68\" (UniqueName: \"kubernetes.io/projected/40244b57-1099-4853-be6e-ef487a3abdd4-kube-api-access-8bv68\") pod \"managed-serviceaccount-addon-agent-ff97ccffd-zwrlr\" (UID: \"40244b57-1099-4853-be6e-ef487a3abdd4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"
Jul 24 17:41:03.866976 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.866912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40244b57-1099-4853-be6e-ef487a3abdd4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-ff97ccffd-zwrlr\" (UID: \"40244b57-1099-4853-be6e-ef487a3abdd4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"
Jul 24 17:41:03.869228 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.869211    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40244b57-1099-4853-be6e-ef487a3abdd4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-ff97ccffd-zwrlr\" (UID: \"40244b57-1099-4853-be6e-ef487a3abdd4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"
Jul 24 17:41:03.877575 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.877554    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bv68\" (UniqueName: \"kubernetes.io/projected/40244b57-1099-4853-be6e-ef487a3abdd4-kube-api-access-8bv68\") pod \"managed-serviceaccount-addon-agent-ff97ccffd-zwrlr\" (UID: \"40244b57-1099-4853-be6e-ef487a3abdd4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"
Jul 24 17:41:03.931634 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.931614    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"
Jul 24 17:41:03.967707 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.967678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c421f0aa-be5d-4fa9-8b3a-40db518783a3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qgzlf\" (UID: \"c421f0aa-be5d-4fa9-8b3a-40db518783a3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"
Jul 24 17:41:03.973500 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.973452    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:41:03.975102 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:03.975073    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq" event={"ID":"03625b16-050a-4640-8f1b-b8071d53cdc8","Type":"ContainerStarted","Data":"3ae74c20a9f43a6fa8886a364354f8843a35c6320fded4c195f81c2e185356b2"}
Jul 24 17:41:04.008363 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:04.008316    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-s8skq" podStartSLOduration=33.852817704 podStartE2EDuration="35.008298687s" totalImagesPullingTime="1.155480983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:04.007957296 +0000 UTC m=+71.212649094" watchObservedRunningTime="2026-07-24 17:41:04.008298687 +0000 UTC m=+71.212990486"
Jul 24 17:41:04.056928 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:04.056899    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr"]
Jul 24 17:41:04.059998 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:41:04.059975    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40244b57_1099_4853_be6e_ef487a3abdd4.slice/crio-5824f7b1a8b249e736ecc783861cf5f53999c26e9e2f6f53dad2d05d78ac05cd WatchSource:0}: Error finding container 5824f7b1a8b249e736ecc783861cf5f53999c26e9e2f6f53dad2d05d78ac05cd: Status 404 returned error can't find the container with id 5824f7b1a8b249e736ecc783861cf5f53999c26e9e2f6f53dad2d05d78ac05cd
Jul 24 17:41:04.688011 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:04.687984    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-pkshd\""
Jul 24 17:41:04.968619 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:04.968478    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: failed to sync secret cache: timed out waiting for the condition
Jul 24 17:41:04.968619 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:04.968574    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c421f0aa-be5d-4fa9-8b3a-40db518783a3-tls-certificates podName:c421f0aa-be5d-4fa9-8b3a-40db518783a3 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:05.468553069 +0000 UTC m=+72.673244847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c421f0aa-be5d-4fa9-8b3a-40db518783a3-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-qgzlf" (UID: "c421f0aa-be5d-4fa9-8b3a-40db518783a3") : failed to sync secret cache: timed out waiting for the condition
Jul 24 17:41:04.984873 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:04.984838    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr" event={"ID":"40244b57-1099-4853-be6e-ef487a3abdd4","Type":"ContainerStarted","Data":"5824f7b1a8b249e736ecc783861cf5f53999c26e9e2f6f53dad2d05d78ac05cd"}
Jul 24 17:41:05.086712 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:05.086681    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 24 17:41:05.479547 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:05.479516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c421f0aa-be5d-4fa9-8b3a-40db518783a3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qgzlf\" (UID: \"c421f0aa-be5d-4fa9-8b3a-40db518783a3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"
Jul 24 17:41:05.482290 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:05.482264    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c421f0aa-be5d-4fa9-8b3a-40db518783a3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qgzlf\" (UID: \"c421f0aa-be5d-4fa9-8b3a-40db518783a3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"
Jul 24 17:41:05.528334 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:05.528308    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"
Jul 24 17:41:06.634638 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:06.634609    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"]
Jul 24 17:41:06.638498 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:41:06.638475    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc421f0aa_be5d_4fa9_8b3a_40db518783a3.slice/crio-aee568a69a9799fdcdcbb2f2c92a063e565079350d75c4b50e95934260575ced WatchSource:0}: Error finding container aee568a69a9799fdcdcbb2f2c92a063e565079350d75c4b50e95934260575ced: Status 404 returned error can't find the container with id aee568a69a9799fdcdcbb2f2c92a063e565079350d75c4b50e95934260575ced
Jul 24 17:41:06.993709 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:06.993673    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf" event={"ID":"c421f0aa-be5d-4fa9-8b3a-40db518783a3","Type":"ContainerStarted","Data":"aee568a69a9799fdcdcbb2f2c92a063e565079350d75c4b50e95934260575ced"}
Jul 24 17:41:06.997591 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:06.997555    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr" event={"ID":"40244b57-1099-4853-be6e-ef487a3abdd4","Type":"ContainerStarted","Data":"e70bd73ff6eeaac74d00cabbb7317cca69c613882507f754ce35db7f8240ba58"}
Jul 24 17:41:07.012713 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:07.012669    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff97ccffd-zwrlr" podStartSLOduration=1.514842526 podStartE2EDuration="4.012653842s" totalImagesPullingTime="2.497811316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:07.012177566 +0000 UTC m=+74.216869364" watchObservedRunningTime="2026-07-24 17:41:07.012653842 +0000 UTC m=+74.217345638"
Jul 24 17:41:07.838678 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:07.838646    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:41:07.839286 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:07.839088    2562 scope.go:117] "RemoveContainer" containerID="00759e559141f4ffe5c56f256148abcdc26fdf42420720dee84d3cb5b4d246f7"
Jul 24 17:41:07.839362 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:07.839276    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-8gfzp_openshift-console-operator(556df0ae-3850-45c5-a889-d36921adf671)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671"
Jul 24 17:41:07.955685 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:07.955661    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:41:08.001131 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:08.001106    2562 scope.go:117] "RemoveContainer" containerID="00759e559141f4ffe5c56f256148abcdc26fdf42420720dee84d3cb5b4d246f7"
Jul 24 17:41:08.001246 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:08.001234    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-8gfzp_openshift-console-operator(556df0ae-3850-45c5-a889-d36921adf671)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671"
Jul 24 17:41:08.861229 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:08.861199    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mf7bb"
Jul 24 17:41:08.944554 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:08.944529    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7jczg"
Jul 24 17:41:09.006254 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:09.006223    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf" event={"ID":"c421f0aa-be5d-4fa9-8b3a-40db518783a3","Type":"ContainerStarted","Data":"ce729f9602c5fd2737e2c2a5d87e24ced70676e7d751344a68e14c81137aa196"}
Jul 24 17:41:09.006433 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:09.006417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"
Jul 24 17:41:09.010727 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:09.010700    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf"
Jul 24 17:41:09.020310 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:09.020271    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qgzlf" podStartSLOduration=4.415024665 podStartE2EDuration="6.020257266s" totalImagesPullingTime="1.605232601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:09.018920758 +0000 UTC m=+76.223612554" watchObservedRunningTime="2026-07-24 17:41:09.020257266 +0000 UTC m=+76.224949062"
Jul 24 17:41:10.617535 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:10.617505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:41:10.619658 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:10.619639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae081de-f739-45a4-a66e-7f9d9e56f35b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xcppd\" (UID: \"2ae081de-f739-45a4-a66e-7f9d9e56f35b\") " pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:41:10.886752 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:10.886691    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g9gf7\""
Jul 24 17:41:10.894921 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:10.894903    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xcppd"
Jul 24 17:41:11.058560 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:11.058536    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xcppd"]
Jul 24 17:41:11.059340 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:41:11.059315    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae081de_f739_45a4_a66e_7f9d9e56f35b.slice/crio-ba6e16fe1e5f8ea6bce73ac5a4837eaff478c11d58b50ae8819335d04a02b1bc WatchSource:0}: Error finding container ba6e16fe1e5f8ea6bce73ac5a4837eaff478c11d58b50ae8819335d04a02b1bc: Status 404 returned error can't find the container with id ba6e16fe1e5f8ea6bce73ac5a4837eaff478c11d58b50ae8819335d04a02b1bc
Jul 24 17:41:12.020352 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:12.020311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xcppd" event={"ID":"2ae081de-f739-45a4-a66e-7f9d9e56f35b","Type":"ContainerStarted","Data":"ba6e16fe1e5f8ea6bce73ac5a4837eaff478c11d58b50ae8819335d04a02b1bc"}
Jul 24 17:41:15.053741 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.053703    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8jqms"]
Jul 24 17:41:15.072109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.072087    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.074827 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.074805    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 17:41:15.075141 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.075121    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sft4t\""
Jul 24 17:41:15.075246 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.075141    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 17:41:15.075246 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.075141    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 17:41:15.075591 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.075576    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 17:41:15.152867 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.152841    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-tls\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.153090 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.152875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35eb8864-a07c-4198-951e-d240bd5036be-root\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.153090 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.152939    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-textfile\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.153090 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.152975    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.153090 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.153003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.153090 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.153044    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-wtmp\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.153090 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.153078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35eb8864-a07c-4198-951e-d240bd5036be-metrics-client-ca\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.153351 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.153096    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxt6s\" (UniqueName: \"kubernetes.io/projected/35eb8864-a07c-4198-951e-d240bd5036be-kube-api-access-pxt6s\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.153351 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.153128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35eb8864-a07c-4198-951e-d240bd5036be-sys\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254054 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-tls\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254219 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254067    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35eb8864-a07c-4198-951e-d240bd5036be-root\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254219 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-textfile\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254219 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254219 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254219 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254158    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-wtmp\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254219 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254166    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35eb8864-a07c-4198-951e-d240bd5036be-root\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254219 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254178    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35eb8864-a07c-4198-951e-d240bd5036be-metrics-client-ca\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254219 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:15.254182    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 17:41:15.254219 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxt6s\" (UniqueName: \"kubernetes.io/projected/35eb8864-a07c-4198-951e-d240bd5036be-kube-api-access-pxt6s\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254702 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:15.254249    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-tls podName:35eb8864-a07c-4198-951e-d240bd5036be nodeName:}" failed. No retries permitted until 2026-07-24 17:41:15.754227775 +0000 UTC m=+82.958919551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-tls") pod "node-exporter-8jqms" (UID: "35eb8864-a07c-4198-951e-d240bd5036be") : secret "node-exporter-tls" not found
Jul 24 17:41:15.254702 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35eb8864-a07c-4198-951e-d240bd5036be-sys\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254702 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254379    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35eb8864-a07c-4198-951e-d240bd5036be-sys\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254702 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254570    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-wtmp\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.254994 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.254969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-textfile\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.255116 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.255079    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35eb8864-a07c-4198-951e-d240bd5036be-metrics-client-ca\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.255208 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.255184    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.256744 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.256720    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.262506 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.262484    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxt6s\" (UniqueName: \"kubernetes.io/projected/35eb8864-a07c-4198-951e-d240bd5036be-kube-api-access-pxt6s\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.274570 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.274546    2562 patch_prober.go:28] interesting pod/image-registry-55d8dbcb57-tglbk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 17:41:15.274682 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.274594    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk" podUID="e6011f48-0283-430a-8d6a-e81ce31eec5b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 17:41:15.759001 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.758965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-tls\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.761601 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.761578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35eb8864-a07c-4198-951e-d240bd5036be-node-exporter-tls\") pod \"node-exporter-8jqms\" (UID: \"35eb8864-a07c-4198-951e-d240bd5036be\") " pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.982337 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:15.982305    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8jqms"
Jul 24 17:41:15.991700 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:41:15.991674    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35eb8864_a07c_4198_951e_d240bd5036be.slice/crio-742788021cb167debbf27945e8710a019a874e3e72be6f2c1e97846c77f1a86e WatchSource:0}: Error finding container 742788021cb167debbf27945e8710a019a874e3e72be6f2c1e97846c77f1a86e: Status 404 returned error can't find the container with id 742788021cb167debbf27945e8710a019a874e3e72be6f2c1e97846c77f1a86e
Jul 24 17:41:16.036817 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:16.036791    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xcppd" event={"ID":"2ae081de-f739-45a4-a66e-7f9d9e56f35b","Type":"ContainerStarted","Data":"e3ae95c92bdb4dfa312422f3646afc1b74028adcb2e0ba35154ddde81cdee6e8"}
Jul 24 17:41:16.037806 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:16.037778    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jqms" event={"ID":"35eb8864-a07c-4198-951e-d240bd5036be","Type":"ContainerStarted","Data":"742788021cb167debbf27945e8710a019a874e3e72be6f2c1e97846c77f1a86e"}
Jul 24 17:41:16.932517 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:16.932219    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:41:18.049818 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:18.049777    2562 generic.go:356] "Generic (PLEG): container finished" podID="35eb8864-a07c-4198-951e-d240bd5036be" containerID="7a9d6a1f19142fe90acb248e43894dac157f8ddde2ce1281dfbea62152d96fbb" exitCode=0
Jul 24 17:41:18.050269 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:18.049837    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jqms" event={"ID":"35eb8864-a07c-4198-951e-d240bd5036be","Type":"ContainerDied","Data":"7a9d6a1f19142fe90acb248e43894dac157f8ddde2ce1281dfbea62152d96fbb"}
Jul 24 17:41:20.060733 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:20.060685    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jqms" event={"ID":"35eb8864-a07c-4198-951e-d240bd5036be","Type":"ContainerStarted","Data":"27755c04ede608c7a6ceca8968c625f10d2826f1c11d9945cd384d31a4d19d6c"}
Jul 24 17:41:20.061216 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:20.060771    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jqms" event={"ID":"35eb8864-a07c-4198-951e-d240bd5036be","Type":"ContainerStarted","Data":"c2013c66749518789e848a66224bb40cacdd0fba688c7c188138e0a9b4618916"}
Jul 24 17:41:20.062737 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:20.062712    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xcppd" event={"ID":"2ae081de-f739-45a4-a66e-7f9d9e56f35b","Type":"ContainerStarted","Data":"10462f77eb1597c6f47a53c910776b7431eb6ea40ff76e90aa3f0197a9d92aaf"}
Jul 24 17:41:20.082205 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:20.082152    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8jqms" podStartSLOduration=3.862248696 podStartE2EDuration="5.082132933s" totalImagesPullingTime="1.141971426s" totalInitContainerRuntime="77.912811ms" isStatefulPod=true podCreationTimestamp="2026-07-24 17:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:20.080097567 +0000 UTC m=+87.284789369" watchObservedRunningTime="2026-07-24 17:41:20.082132933 +0000 UTC m=+87.286824731"
Jul 24 17:41:20.402128 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:20.402063    2562 scope.go:117] "RemoveContainer" containerID="00759e559141f4ffe5c56f256148abcdc26fdf42420720dee84d3cb5b4d246f7"
Jul 24 17:41:20.402283 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:20.402246    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-8gfzp_openshift-console-operator(556df0ae-3850-45c5-a889-d36921adf671)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" podUID="556df0ae-3850-45c5-a889-d36921adf671"
Jul 24 17:41:23.076941 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:23.076909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xcppd" event={"ID":"2ae081de-f739-45a4-a66e-7f9d9e56f35b","Type":"ContainerStarted","Data":"c8a3406a40f732cd3045fee310681bf08abfdf0d4bbeedb651a30fb14ff99e9e"}
Jul 24 17:41:23.095010 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:23.094936    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xcppd" podStartSLOduration=33.429718421 podStartE2EDuration="45.094917641s" totalImagesPullingTime="11.66519922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:23.094023791 +0000 UTC m=+90.298715588" watchObservedRunningTime="2026-07-24 17:41:23.094917641 +0000 UTC m=+90.299609437"
Jul 24 17:41:27.347341 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:27.347312    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55d8dbcb57-tglbk"]
Jul 24 17:41:33.403068 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:33.403034    2562 scope.go:117] "RemoveContainer" containerID="00759e559141f4ffe5c56f256148abcdc26fdf42420720dee84d3cb5b4d246f7"
Jul 24 17:41:34.116328 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.116298    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:41:34.116477 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.116360    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp" event={"ID":"556df0ae-3850-45c5-a889-d36921adf671","Type":"ContainerStarted","Data":"46ce502928af6d1755d09b41a47be40df423c44360ac82bc7ddc12ffc222da0e"}
Jul 24 17:41:34.116643 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.116618    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:41:34.131753 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.131727    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-8gfzp"
Jul 24 17:41:34.310153 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.310122    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-6s4h4"]
Jul 24 17:41:34.313894 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.313855    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-6s4h4"
Jul 24 17:41:34.316557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.316531    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 17:41:34.316693 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.316590    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 17:41:34.316693 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.316634    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wb8xk\""
Jul 24 17:41:34.322466 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.322441    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-6s4h4"]
Jul 24 17:41:34.404346 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.404271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx9cn\" (UniqueName: \"kubernetes.io/projected/2ccc92be-bd5c-41c7-84e9-6b2e76f71654-kube-api-access-zx9cn\") pod \"downloads-7c97dc4555-6s4h4\" (UID: \"2ccc92be-bd5c-41c7-84e9-6b2e76f71654\") " pod="openshift-console/downloads-7c97dc4555-6s4h4"
Jul 24 17:41:34.505510 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.505481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx9cn\" (UniqueName: \"kubernetes.io/projected/2ccc92be-bd5c-41c7-84e9-6b2e76f71654-kube-api-access-zx9cn\") pod \"downloads-7c97dc4555-6s4h4\" (UID: \"2ccc92be-bd5c-41c7-84e9-6b2e76f71654\") " pod="openshift-console/downloads-7c97dc4555-6s4h4"
Jul 24 17:41:34.516345 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.516318    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx9cn\" (UniqueName: \"kubernetes.io/projected/2ccc92be-bd5c-41c7-84e9-6b2e76f71654-kube-api-access-zx9cn\") pod \"downloads-7c97dc4555-6s4h4\" (UID: \"2ccc92be-bd5c-41c7-84e9-6b2e76f71654\") " pod="openshift-console/downloads-7c97dc4555-6s4h4"
Jul 24 17:41:34.624347 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.624315    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-6s4h4"
Jul 24 17:41:34.748745 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:34.748720    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-6s4h4"]
Jul 24 17:41:34.751520 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:41:34.751489    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ccc92be_bd5c_41c7_84e9_6b2e76f71654.slice/crio-96807a9e1f8c7f2ef1025e7d0d916b84b3eaf8566eba9467f44d46a6e46d3bb2 WatchSource:0}: Error finding container 96807a9e1f8c7f2ef1025e7d0d916b84b3eaf8566eba9467f44d46a6e46d3bb2: Status 404 returned error can't find the container with id 96807a9e1f8c7f2ef1025e7d0d916b84b3eaf8566eba9467f44d46a6e46d3bb2
Jul 24 17:41:35.121304 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:35.121262    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-6s4h4" event={"ID":"2ccc92be-bd5c-41c7-84e9-6b2e76f71654","Type":"ContainerStarted","Data":"96807a9e1f8c7f2ef1025e7d0d916b84b3eaf8566eba9467f44d46a6e46d3bb2"}
Jul 24 17:41:39.827046 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.826997    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-694f558b59-ckv6t"]
Jul 24 17:41:39.872288 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.872256    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-694f558b59-ckv6t"]
Jul 24 17:41:39.872436 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.872387    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:39.874837 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.874807    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 24 17:41:39.875720 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.875553    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 24 17:41:39.875720 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.875566    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 24 17:41:39.875720 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.875598    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 24 17:41:39.875720 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.875604    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 24 17:41:39.875720 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.875621    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2bmd2\""
Jul 24 17:41:39.947211 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.947183    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvgsj\" (UniqueName: \"kubernetes.io/projected/558cd29c-64f2-4ca4-a07f-515ef5391d08-kube-api-access-jvgsj\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:39.947346 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.947237    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-config\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:39.947346 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.947281    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-oauth-config\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:39.947346 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.947333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-service-ca\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:39.947500 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.947363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-oauth-serving-cert\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:39.947500 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:39.947390    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-serving-cert\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.048573 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.048542    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-oauth-serving-cert\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.048701 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.048579    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-serving-cert\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.048701 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.048622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvgsj\" (UniqueName: \"kubernetes.io/projected/558cd29c-64f2-4ca4-a07f-515ef5391d08-kube-api-access-jvgsj\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.048701 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.048651    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-config\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.048701 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.048697    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-oauth-config\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.048945 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.048727    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-service-ca\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.049339 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.049308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-oauth-serving-cert\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.049489 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.049372    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-service-ca\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.049667 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.049636    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-config\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.051521 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.051494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-serving-cert\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.051626 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.051608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-oauth-config\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.062395 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.062371    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvgsj\" (UniqueName: \"kubernetes.io/projected/558cd29c-64f2-4ca4-a07f-515ef5391d08-kube-api-access-jvgsj\") pod \"console-694f558b59-ckv6t\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") " pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.183992 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.183919    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:40.356136 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:40.356104    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-694f558b59-ckv6t"]
Jul 24 17:41:40.359401 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:41:40.359371    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558cd29c_64f2_4ca4_a07f_515ef5391d08.slice/crio-1082623e1717eb25ff2d2fcb888650090606ed6daa0fee7d600260e19e2670f1 WatchSource:0}: Error finding container 1082623e1717eb25ff2d2fcb888650090606ed6daa0fee7d600260e19e2670f1: Status 404 returned error can't find the container with id 1082623e1717eb25ff2d2fcb888650090606ed6daa0fee7d600260e19e2670f1
Jul 24 17:41:41.152797 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:41.152754    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694f558b59-ckv6t" event={"ID":"558cd29c-64f2-4ca4-a07f-515ef5391d08","Type":"ContainerStarted","Data":"1082623e1717eb25ff2d2fcb888650090606ed6daa0fee7d600260e19e2670f1"}
Jul 24 17:41:44.168794 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:44.168747    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694f558b59-ckv6t" event={"ID":"558cd29c-64f2-4ca4-a07f-515ef5391d08","Type":"ContainerStarted","Data":"4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd"}
Jul 24 17:41:44.194769 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:44.194715    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-694f558b59-ckv6t" podStartSLOduration=2.205300353 podStartE2EDuration="5.194699879s" totalImagesPullingTime="2.989399526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:44.192728839 +0000 UTC m=+111.397420637" watchObservedRunningTime="2026-07-24 17:41:44.194699879 +0000 UTC m=+111.399391676"
Jul 24 17:41:49.212536 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.212490    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68bdb49bd8-q7khm"]
Jul 24 17:41:49.326342 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.326305    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68bdb49bd8-q7khm"]
Jul 24 17:41:49.326521 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.326419    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.334241 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.334213    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 24 17:41:49.428857 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.428825    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gz7k\" (UniqueName: \"kubernetes.io/projected/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-kube-api-access-9gz7k\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.429027 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.428896    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-oauth-config\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.429027 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.428971    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-config\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.429130 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.429029    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-service-ca\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.429130 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.429060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-serving-cert\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.429130 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.429099    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-oauth-serving-cert\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.429257 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.429132    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-trusted-ca-bundle\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.530314 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.530243    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-oauth-config\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.530314 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.530285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-config\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.530314 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.530311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-service-ca\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.530577 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.530329    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-serving-cert\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.530577 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.530351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-oauth-serving-cert\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.530577 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.530386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-trusted-ca-bundle\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.530577 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.530444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gz7k\" (UniqueName: \"kubernetes.io/projected/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-kube-api-access-9gz7k\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.531075 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.531025    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-config\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.531210 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.531113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-oauth-serving-cert\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.531275 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.531208    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-service-ca\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.531570 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.531540    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-trusted-ca-bundle\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.533374 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.533341    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-serving-cert\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.533491 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.533452    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-oauth-config\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.540040 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.540017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gz7k\" (UniqueName: \"kubernetes.io/projected/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-kube-api-access-9gz7k\") pod \"console-68bdb49bd8-q7khm\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") " pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:49.638197 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:49.638163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:50.184845 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:50.184805    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:50.185070 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:50.184893    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:50.190589 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:50.190564    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:50.199025 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:50.199001    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:41:51.399149 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:51.399125    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68bdb49bd8-q7khm"]
Jul 24 17:41:51.406231 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:41:51.406199    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f6dae2_5d1c_4b9c_86bf_043e5e3882ff.slice/crio-8635413c3fc9de32ac90c3b4ee8eb2423d6e4ab8d3b70a69bb1527a8944a6811 WatchSource:0}: Error finding container 8635413c3fc9de32ac90c3b4ee8eb2423d6e4ab8d3b70a69bb1527a8944a6811: Status 404 returned error can't find the container with id 8635413c3fc9de32ac90c3b4ee8eb2423d6e4ab8d3b70a69bb1527a8944a6811
Jul 24 17:41:52.205516 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.205466    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-6s4h4" event={"ID":"2ccc92be-bd5c-41c7-84e9-6b2e76f71654","Type":"ContainerStarted","Data":"30cd5af714a4d3f6c4e0164e59c677db11396d8c7646c056f1cfc9c1497fa3e6"}
Jul 24 17:41:52.205711 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.205576    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-6s4h4"
Jul 24 17:41:52.207762 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.207727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bdb49bd8-q7khm" event={"ID":"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff","Type":"ContainerStarted","Data":"2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6"}
Jul 24 17:41:52.207913 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.207773    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bdb49bd8-q7khm" event={"ID":"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff","Type":"ContainerStarted","Data":"8635413c3fc9de32ac90c3b4ee8eb2423d6e4ab8d3b70a69bb1527a8944a6811"}
Jul 24 17:41:52.221828 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.221798    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-6s4h4"
Jul 24 17:41:52.224166 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.224118    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-6s4h4" podStartSLOduration=1.612178071 podStartE2EDuration="18.224102957s" totalImagesPullingTime="16.611924886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:52.222527283 +0000 UTC m=+119.427219089" watchObservedRunningTime="2026-07-24 17:41:52.224102957 +0000 UTC m=+119.428794755"
Jul 24 17:41:52.258535 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.258478    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68bdb49bd8-q7khm" podStartSLOduration=3.258460119 podStartE2EDuration="3.258460119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:52.256984595 +0000 UTC m=+119.461676394" watchObservedRunningTime="2026-07-24 17:41:52.258460119 +0000 UTC m=+119.463151916"
Jul 24 17:41:52.371665 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.371613    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk" podUID="e6011f48-0283-430a-8d6a-e81ce31eec5b" containerName="registry" containerID="cri-o://df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637" gracePeriod=30
Jul 24 17:41:52.645271 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.645236    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:41:52.761080 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.760983    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-image-registry-private-configuration\") pod \"e6011f48-0283-430a-8d6a-e81ce31eec5b\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") "
Jul 24 17:41:52.761080 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.761048    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6011f48-0283-430a-8d6a-e81ce31eec5b-ca-trust-extracted\") pod \"e6011f48-0283-430a-8d6a-e81ce31eec5b\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") "
Jul 24 17:41:52.761308 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.761219    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-trusted-ca\") pod \"e6011f48-0283-430a-8d6a-e81ce31eec5b\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") "
Jul 24 17:41:52.761379 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.761313    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9296j\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-kube-api-access-9296j\") pod \"e6011f48-0283-430a-8d6a-e81ce31eec5b\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") "
Jul 24 17:41:52.761379 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.761362    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") pod \"e6011f48-0283-430a-8d6a-e81ce31eec5b\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") "
Jul 24 17:41:52.761488 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.761399    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-bound-sa-token\") pod \"e6011f48-0283-430a-8d6a-e81ce31eec5b\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") "
Jul 24 17:41:52.761488 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.761460    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-certificates\") pod \"e6011f48-0283-430a-8d6a-e81ce31eec5b\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") "
Jul 24 17:41:52.761585 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.761501    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-installation-pull-secrets\") pod \"e6011f48-0283-430a-8d6a-e81ce31eec5b\" (UID: \"e6011f48-0283-430a-8d6a-e81ce31eec5b\") "
Jul 24 17:41:52.761637 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.761585    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e6011f48-0283-430a-8d6a-e81ce31eec5b" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:41:52.762849 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.761780    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-trusted-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:41:52.762849 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.762217    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e6011f48-0283-430a-8d6a-e81ce31eec5b" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:41:52.764229 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.764178    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-kube-api-access-9296j" (OuterVolumeSpecName: "kube-api-access-9296j") pod "e6011f48-0283-430a-8d6a-e81ce31eec5b" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b"). InnerVolumeSpecName "kube-api-access-9296j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:41:52.764466 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.764392    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e6011f48-0283-430a-8d6a-e81ce31eec5b" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:41:52.765095 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.765041    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e6011f48-0283-430a-8d6a-e81ce31eec5b" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:41:52.765384 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.765342    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e6011f48-0283-430a-8d6a-e81ce31eec5b" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:41:52.765548 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.765520    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e6011f48-0283-430a-8d6a-e81ce31eec5b" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:41:52.772813 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.772784    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6011f48-0283-430a-8d6a-e81ce31eec5b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e6011f48-0283-430a-8d6a-e81ce31eec5b" (UID: "e6011f48-0283-430a-8d6a-e81ce31eec5b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:41:52.862893 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.862840    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-bound-sa-token\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:41:52.862893 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.862873    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-certificates\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:41:52.862893 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.862903    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-installation-pull-secrets\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:41:52.863113 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.862914    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6011f48-0283-430a-8d6a-e81ce31eec5b-image-registry-private-configuration\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:41:52.863113 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.862928    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6011f48-0283-430a-8d6a-e81ce31eec5b-ca-trust-extracted\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:41:52.863113 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.862936    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9296j\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-kube-api-access-9296j\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:41:52.863113 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:52.862944    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6011f48-0283-430a-8d6a-e81ce31eec5b-registry-tls\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:41:53.215070 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.215038    2562 generic.go:356] "Generic (PLEG): container finished" podID="e6011f48-0283-430a-8d6a-e81ce31eec5b" containerID="df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637" exitCode=0
Jul 24 17:41:53.215195 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.215104    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk"
Jul 24 17:41:53.215195 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.215175    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk" event={"ID":"e6011f48-0283-430a-8d6a-e81ce31eec5b","Type":"ContainerDied","Data":"df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637"}
Jul 24 17:41:53.215312 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.215219    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d8dbcb57-tglbk" event={"ID":"e6011f48-0283-430a-8d6a-e81ce31eec5b","Type":"ContainerDied","Data":"6dd27b86e50518ad28e58cbabea2881224e9e1d79de67bca52b4e1e16dafc28b"}
Jul 24 17:41:53.215312 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.215248    2562 scope.go:117] "RemoveContainer" containerID="df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637"
Jul 24 17:41:53.228222 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.228198    2562 scope.go:117] "RemoveContainer" containerID="df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637"
Jul 24 17:41:53.228620 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:53.228559    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637\": container with ID starting with df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637 not found: ID does not exist" containerID="df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637"
Jul 24 17:41:53.228727 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.228606    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637"} err="failed to get container status \"df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637\": rpc error: code = NotFound desc = could not find container \"df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637\": container with ID starting with df3be43480f93feecdd3ec33c1eea6154cbe2998332ef7892834c00749ca4637 not found: ID does not exist"
Jul 24 17:41:53.241945 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.241919    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55d8dbcb57-tglbk"]
Jul 24 17:41:53.245782 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.245761    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55d8dbcb57-tglbk"]
Jul 24 17:41:53.413710 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:53.413673    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6011f48-0283-430a-8d6a-e81ce31eec5b" path="/var/lib/kubelet/pods/e6011f48-0283-430a-8d6a-e81ce31eec5b/volumes"
Jul 24 17:41:53.692691 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:53.692642    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:41:54.221199 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:54.221164    2562 generic.go:356] "Generic (PLEG): container finished" podID="d66bc6dd-82ce-473f-8aba-de7d3929b6bb" containerID="21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db" exitCode=0
Jul 24 17:41:54.221489 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:54.221241    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m7799" event={"ID":"d66bc6dd-82ce-473f-8aba-de7d3929b6bb","Type":"ContainerDied","Data":"21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db"}
Jul 24 17:41:54.221685 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:54.221664    2562 scope.go:117] "RemoveContainer" containerID="21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db"
Jul 24 17:41:54.224177 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:54.224151    2562 generic.go:356] "Generic (PLEG): container finished" podID="65571cc3-f16a-4cfa-a73c-50380be899c2" containerID="1f57a71306fc3d131a55f4fa3ea2cb3c9a787b22a96bd8ddd01923861a9197d5" exitCode=0
Jul 24 17:41:54.224294 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:54.224227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2" event={"ID":"65571cc3-f16a-4cfa-a73c-50380be899c2","Type":"ContainerDied","Data":"1f57a71306fc3d131a55f4fa3ea2cb3c9a787b22a96bd8ddd01923861a9197d5"}
Jul 24 17:41:54.224633 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:54.224615    2562 scope.go:117] "RemoveContainer" containerID="1f57a71306fc3d131a55f4fa3ea2cb3c9a787b22a96bd8ddd01923861a9197d5"
Jul 24 17:41:55.231430 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:55.231385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m7799" event={"ID":"d66bc6dd-82ce-473f-8aba-de7d3929b6bb","Type":"ContainerStarted","Data":"68a453567027f4bba40506e119967b39cad5550463517d54f781144c3b929dcd"}
Jul 24 17:41:55.233653 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:55.233625    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-nblq2" event={"ID":"65571cc3-f16a-4cfa-a73c-50380be899c2","Type":"ContainerStarted","Data":"5910833129dac8c503cbca2f7c4401e2dd1f92c5a1422261733b6877e3af837e"}
Jul 24 17:41:58.659715 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:58.659676    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:41:58.660152 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:58.659769    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:41:58.919221 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:41:58.919143    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:41:59.639239 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:59.639191    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:59.639462 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:59.639276    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:41:59.644939 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:41:59.644915    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:42:00.258570 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:00.258540    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:42:00.310744 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:00.310713    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-694f558b59-ckv6t"]
Jul 24 17:42:03.703586 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:42:03.703552    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:42:09.298080 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:09.298043    2562 generic.go:356] "Generic (PLEG): container finished" podID="8dd06a5f-d5af-4e05-bd87-afdbf92c7b43" containerID="5c7f1439b647acc12d6f46dca07499f34951b900542e5cebd3c9bc15306d0e55" exitCode=0
Jul 24 17:42:09.298483 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:09.298119    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q" event={"ID":"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43","Type":"ContainerDied","Data":"5c7f1439b647acc12d6f46dca07499f34951b900542e5cebd3c9bc15306d0e55"}
Jul 24 17:42:09.298527 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:09.298501    2562 scope.go:117] "RemoveContainer" containerID="5c7f1439b647acc12d6f46dca07499f34951b900542e5cebd3c9bc15306d0e55"
Jul 24 17:42:10.303831 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:10.303793    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q" event={"ID":"8dd06a5f-d5af-4e05-bd87-afdbf92c7b43","Type":"ContainerStarted","Data":"2765d220af66bce1734479438432b2f03f09410aed163e65f167bb2743491771"}
Jul 24 17:42:13.741512 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:42:13.741470    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:42:13.882551 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:42:13.882513    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:42:23.777528 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:42:23.777481    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:42:25.331119 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.331072    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-694f558b59-ckv6t" podUID="558cd29c-64f2-4ca4-a07f-515ef5391d08" containerName="console" containerID="cri-o://4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd" gracePeriod=15
Jul 24 17:42:25.614535 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.614467    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:42:25.731849 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.731819    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-service-ca\") pod \"558cd29c-64f2-4ca4-a07f-515ef5391d08\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") "
Jul 24 17:42:25.731975 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.731895    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvgsj\" (UniqueName: \"kubernetes.io/projected/558cd29c-64f2-4ca4-a07f-515ef5391d08-kube-api-access-jvgsj\") pod \"558cd29c-64f2-4ca4-a07f-515ef5391d08\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") "
Jul 24 17:42:25.731975 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.731917    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-config\") pod \"558cd29c-64f2-4ca4-a07f-515ef5391d08\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") "
Jul 24 17:42:25.731975 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.731937    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-oauth-serving-cert\") pod \"558cd29c-64f2-4ca4-a07f-515ef5391d08\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") "
Jul 24 17:42:25.731975 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.731964    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-oauth-config\") pod \"558cd29c-64f2-4ca4-a07f-515ef5391d08\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") "
Jul 24 17:42:25.732113 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.732034    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-serving-cert\") pod \"558cd29c-64f2-4ca4-a07f-515ef5391d08\" (UID: \"558cd29c-64f2-4ca4-a07f-515ef5391d08\") "
Jul 24 17:42:25.732359 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.732328    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-service-ca" (OuterVolumeSpecName: "service-ca") pod "558cd29c-64f2-4ca4-a07f-515ef5391d08" (UID: "558cd29c-64f2-4ca4-a07f-515ef5391d08"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:25.732430 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.732358    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-config" (OuterVolumeSpecName: "console-config") pod "558cd29c-64f2-4ca4-a07f-515ef5391d08" (UID: "558cd29c-64f2-4ca4-a07f-515ef5391d08"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:25.732465 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.732424    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "558cd29c-64f2-4ca4-a07f-515ef5391d08" (UID: "558cd29c-64f2-4ca4-a07f-515ef5391d08"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:25.734207 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.734180    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "558cd29c-64f2-4ca4-a07f-515ef5391d08" (UID: "558cd29c-64f2-4ca4-a07f-515ef5391d08"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:25.734277 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.734259    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "558cd29c-64f2-4ca4-a07f-515ef5391d08" (UID: "558cd29c-64f2-4ca4-a07f-515ef5391d08"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:25.734373 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.734352    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/558cd29c-64f2-4ca4-a07f-515ef5391d08-kube-api-access-jvgsj" (OuterVolumeSpecName: "kube-api-access-jvgsj") pod "558cd29c-64f2-4ca4-a07f-515ef5391d08" (UID: "558cd29c-64f2-4ca4-a07f-515ef5391d08"). InnerVolumeSpecName "kube-api-access-jvgsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:25.832643 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.832607    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvgsj\" (UniqueName: \"kubernetes.io/projected/558cd29c-64f2-4ca4-a07f-515ef5391d08-kube-api-access-jvgsj\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:42:25.832643 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.832636    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-config\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:42:25.832643 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.832646    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-oauth-serving-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:42:25.832643 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.832654    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-oauth-config\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:42:25.832919 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.832672    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/558cd29c-64f2-4ca4-a07f-515ef5391d08-console-serving-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:42:25.832919 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:25.832681    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/558cd29c-64f2-4ca4-a07f-515ef5391d08-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:42:26.372247 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:26.372214    2562 generic.go:356] "Generic (PLEG): container finished" podID="558cd29c-64f2-4ca4-a07f-515ef5391d08" containerID="4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd" exitCode=0
Jul 24 17:42:26.372646 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:26.372255    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694f558b59-ckv6t"
Jul 24 17:42:26.372646 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:26.372305    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694f558b59-ckv6t" event={"ID":"558cd29c-64f2-4ca4-a07f-515ef5391d08","Type":"ContainerDied","Data":"4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd"}
Jul 24 17:42:26.372646 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:26.372356    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694f558b59-ckv6t" event={"ID":"558cd29c-64f2-4ca4-a07f-515ef5391d08","Type":"ContainerDied","Data":"1082623e1717eb25ff2d2fcb888650090606ed6daa0fee7d600260e19e2670f1"}
Jul 24 17:42:26.372646 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:26.372379    2562 scope.go:117] "RemoveContainer" containerID="4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd"
Jul 24 17:42:26.380981 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:26.380965    2562 scope.go:117] "RemoveContainer" containerID="4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd"
Jul 24 17:42:26.381237 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:42:26.381221    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd\": container with ID starting with 4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd not found: ID does not exist" containerID="4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd"
Jul 24 17:42:26.381282 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:26.381246    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd"} err="failed to get container status \"4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd\": rpc error: code = NotFound desc = could not find container \"4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd\": container with ID starting with 4c47d6ad83b55c556e9f840dea42bb1f06ab5b9ee9a0c12085b3553bcce404fd not found: ID does not exist"
Jul 24 17:42:26.393284 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:26.393258    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-694f558b59-ckv6t"]
Jul 24 17:42:26.403547 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:26.403526    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-694f558b59-ckv6t"]
Jul 24 17:42:27.405869 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:27.405833    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="558cd29c-64f2-4ca4-a07f-515ef5391d08" path="/var/lib/kubelet/pods/558cd29c-64f2-4ca4-a07f-515ef5391d08/volumes"
Jul 24 17:42:28.899483 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:42:28.899450    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:42:33.785624 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:42:33.785581    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:42:43.663204 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.663171    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d64b7c69c-cx46h"]
Jul 24 17:42:43.663758 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.663559    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="558cd29c-64f2-4ca4-a07f-515ef5391d08" containerName="console"
Jul 24 17:42:43.663758 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.663576    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="558cd29c-64f2-4ca4-a07f-515ef5391d08" containerName="console"
Jul 24 17:42:43.663758 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.663611    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6011f48-0283-430a-8d6a-e81ce31eec5b" containerName="registry"
Jul 24 17:42:43.663758 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.663619    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6011f48-0283-430a-8d6a-e81ce31eec5b" containerName="registry"
Jul 24 17:42:43.663758 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.663756    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6011f48-0283-430a-8d6a-e81ce31eec5b" containerName="registry"
Jul 24 17:42:43.664048 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.663772    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="558cd29c-64f2-4ca4-a07f-515ef5391d08" containerName="console"
Jul 24 17:42:43.666935 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.666905    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.679170 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.679139    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d64b7c69c-cx46h"]
Jul 24 17:42:43.765699 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.765672    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-console-config\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.765699 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.765702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-oauth-serving-cert\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.765956 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.765723    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdfn7\" (UniqueName: \"kubernetes.io/projected/bb1e21a6-5747-432c-809c-f614ebd8e211-kube-api-access-xdfn7\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.765956 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.765804    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-trusted-ca-bundle\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.765956 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.765852    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-oauth-config\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.765956 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.765871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-service-ca\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.766131 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.765976    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-serving-cert\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.821512 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:42:43.821475    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:42:43.867147 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.867115    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-trusted-ca-bundle\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.867317 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.867171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-oauth-config\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.867317 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.867194    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-service-ca\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.867317 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.867232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-serving-cert\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.867317 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.867296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-console-config\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.867491 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.867325    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-oauth-serving-cert\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.867491 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.867348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdfn7\" (UniqueName: \"kubernetes.io/projected/bb1e21a6-5747-432c-809c-f614ebd8e211-kube-api-access-xdfn7\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.868080 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.868049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-service-ca\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.868224 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.868094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-trusted-ca-bundle\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.868224 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.868131    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-oauth-serving-cert\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.868224 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.868151    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-console-config\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.869701 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.869682    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-oauth-config\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.869800 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.869783    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-serving-cert\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.875944 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.875922    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdfn7\" (UniqueName: \"kubernetes.io/projected/bb1e21a6-5747-432c-809c-f614ebd8e211-kube-api-access-xdfn7\") pod \"console-7d64b7c69c-cx46h\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") " pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:43.884318 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:42:43.884296    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66bc6dd_82ce_473f_8aba_de7d3929b6bb.slice/crio-21a79b02c0767b8467eff4d64c891938cddaa4b345dc0b42bb461dd2e5b370db.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 17:42:43.976431 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:43.976347    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:44.107398 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:44.107366    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d64b7c69c-cx46h"]
Jul 24 17:42:44.110227 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:42:44.110204    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1e21a6_5747_432c_809c_f614ebd8e211.slice/crio-ae4b0e31f5876a22318198b581b432788e55120dd51a8b314c444a63bd22f2d7 WatchSource:0}: Error finding container ae4b0e31f5876a22318198b581b432788e55120dd51a8b314c444a63bd22f2d7: Status 404 returned error can't find the container with id ae4b0e31f5876a22318198b581b432788e55120dd51a8b314c444a63bd22f2d7
Jul 24 17:42:44.438968 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:44.438934    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d64b7c69c-cx46h" event={"ID":"bb1e21a6-5747-432c-809c-f614ebd8e211","Type":"ContainerStarted","Data":"810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31"}
Jul 24 17:42:44.439128 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:44.438982    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d64b7c69c-cx46h" event={"ID":"bb1e21a6-5747-432c-809c-f614ebd8e211","Type":"ContainerStarted","Data":"ae4b0e31f5876a22318198b581b432788e55120dd51a8b314c444a63bd22f2d7"}
Jul 24 17:42:44.459503 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:44.459457    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d64b7c69c-cx46h" podStartSLOduration=1.459441697 podStartE2EDuration="1.459441697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:42:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:42:44.458178757 +0000 UTC m=+171.662870554" watchObservedRunningTime="2026-07-24 17:42:44.459441697 +0000 UTC m=+171.664133503"
Jul 24 17:42:53.977284 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:53.977252    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:53.977685 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:53.977308    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:53.981862 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:53.981837    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:54.480464 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:54.480436    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:42:54.529127 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:42:54.529093    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68bdb49bd8-q7khm"]
Jul 24 17:43:19.551900 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.551842    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68bdb49bd8-q7khm" podUID="76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" containerName="console" containerID="cri-o://2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6" gracePeriod=15
Jul 24 17:43:19.795596 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.795574    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:43:19.928267 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928202    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-service-ca\") pod \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") "
Jul 24 17:43:19.928267 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928237    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-config\") pod \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") "
Jul 24 17:43:19.928267 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928264    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-serving-cert\") pod \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") "
Jul 24 17:43:19.928480 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928291    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gz7k\" (UniqueName: \"kubernetes.io/projected/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-kube-api-access-9gz7k\") pod \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") "
Jul 24 17:43:19.928480 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928319    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-oauth-serving-cert\") pod \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") "
Jul 24 17:43:19.928480 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928349    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-trusted-ca-bundle\") pod \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") "
Jul 24 17:43:19.928480 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928382    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-oauth-config\") pod \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\" (UID: \"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff\") "
Jul 24 17:43:19.928678 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928643    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-config" (OuterVolumeSpecName: "console-config") pod "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" (UID: "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:43:19.928737 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928593    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-service-ca" (OuterVolumeSpecName: "service-ca") pod "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" (UID: "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:43:19.928823 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928798    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" (UID: "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:43:19.928948 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.928822    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" (UID: "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:43:19.930448 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.930419    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" (UID: "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:43:19.930546 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.930505    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" (UID: "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:43:19.930546 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:19.930532    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-kube-api-access-9gz7k" (OuterVolumeSpecName: "kube-api-access-9gz7k") pod "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" (UID: "76f6dae2-5d1c-4b9c-86bf-043e5e3882ff"). InnerVolumeSpecName "kube-api-access-9gz7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:43:20.029000 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.028977    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-trusted-ca-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:43:20.029000 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.028996    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-oauth-config\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:43:20.029118 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.029006    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:43:20.029118 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.029014    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-config\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:43:20.029118 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.029021    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-console-serving-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:43:20.029118 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.029029    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gz7k\" (UniqueName: \"kubernetes.io/projected/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-kube-api-access-9gz7k\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:43:20.029118 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.029037    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff-oauth-serving-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:43:20.573326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.573288    2562 generic.go:356] "Generic (PLEG): container finished" podID="76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" containerID="2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6" exitCode=0
Jul 24 17:43:20.573732 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.573340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bdb49bd8-q7khm" event={"ID":"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff","Type":"ContainerDied","Data":"2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6"}
Jul 24 17:43:20.573732 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.573358    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bdb49bd8-q7khm"
Jul 24 17:43:20.573732 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.573382    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bdb49bd8-q7khm" event={"ID":"76f6dae2-5d1c-4b9c-86bf-043e5e3882ff","Type":"ContainerDied","Data":"8635413c3fc9de32ac90c3b4ee8eb2423d6e4ab8d3b70a69bb1527a8944a6811"}
Jul 24 17:43:20.573732 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.573404    2562 scope.go:117] "RemoveContainer" containerID="2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6"
Jul 24 17:43:20.582552 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.582534    2562 scope.go:117] "RemoveContainer" containerID="2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6"
Jul 24 17:43:20.582868 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:43:20.582847    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6\": container with ID starting with 2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6 not found: ID does not exist" containerID="2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6"
Jul 24 17:43:20.582949 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.582904    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6"} err="failed to get container status \"2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6\": rpc error: code = NotFound desc = could not find container \"2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6\": container with ID starting with 2f97aa643455f0a68b7ac45cb24b955ada324f40e64ee2cde38de29a922ea7f6 not found: ID does not exist"
Jul 24 17:43:20.595936 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.595903    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68bdb49bd8-q7khm"]
Jul 24 17:43:20.598892 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:20.598852    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68bdb49bd8-q7khm"]
Jul 24 17:43:21.406467 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:43:21.406420    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" path="/var/lib/kubelet/pods/76f6dae2-5d1c-4b9c-86bf-043e5e3882ff/volumes"
Jul 24 17:44:34.320572 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.320532    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f979b4bcf-wvrcq"]
Jul 24 17:44:34.321111 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.320868    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" containerName="console"
Jul 24 17:44:34.321111 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.320901    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" containerName="console"
Jul 24 17:44:34.321111 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.320965    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f6dae2-5d1c-4b9c-86bf-043e5e3882ff" containerName="console"
Jul 24 17:44:34.322852 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.322835    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.334088 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.334063    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f979b4bcf-wvrcq"]
Jul 24 17:44:34.432807 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.432777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-oauth-config\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.432964 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.432820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-serving-cert\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.432964 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.432899    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-console-config\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.433085 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.432972    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-service-ca\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.433085 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.432999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx2bg\" (UniqueName: \"kubernetes.io/projected/28ef342e-90d9-46f8-82f6-1eb953707974-kube-api-access-mx2bg\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.433085 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.433033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-trusted-ca-bundle\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.433085 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.433054    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-oauth-serving-cert\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.533337 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.533312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-trusted-ca-bundle\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.533337 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.533338    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-oauth-serving-cert\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.533504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.533373    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-oauth-config\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.533504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.533413    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-serving-cert\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.533504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.533444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-console-config\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.533654 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.533528    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-service-ca\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.533654 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.533553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx2bg\" (UniqueName: \"kubernetes.io/projected/28ef342e-90d9-46f8-82f6-1eb953707974-kube-api-access-mx2bg\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.534217 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.534192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-oauth-serving-cert\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.534325 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.534192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-service-ca\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.536039 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.534632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-console-config\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.536288 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.536264    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-trusted-ca-bundle\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.536709 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.536685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-serving-cert\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.536709 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.536701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-oauth-config\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.542698 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.542672    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx2bg\" (UniqueName: \"kubernetes.io/projected/28ef342e-90d9-46f8-82f6-1eb953707974-kube-api-access-mx2bg\") pod \"console-5f979b4bcf-wvrcq\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") " pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.631784 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.631733    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:34.756307 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.756284    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f979b4bcf-wvrcq"]
Jul 24 17:44:34.758149 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:44:34.758111    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ef342e_90d9_46f8_82f6_1eb953707974.slice/crio-3b6c89273f06106257e2c856c47272a721f45c6d82ca5d088974a9429d0d6368 WatchSource:0}: Error finding container 3b6c89273f06106257e2c856c47272a721f45c6d82ca5d088974a9429d0d6368: Status 404 returned error can't find the container with id 3b6c89273f06106257e2c856c47272a721f45c6d82ca5d088974a9429d0d6368
Jul 24 17:44:34.859829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.859257    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f979b4bcf-wvrcq" event={"ID":"28ef342e-90d9-46f8-82f6-1eb953707974","Type":"ContainerStarted","Data":"6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384"}
Jul 24 17:44:34.859829 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.859318    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f979b4bcf-wvrcq" event={"ID":"28ef342e-90d9-46f8-82f6-1eb953707974","Type":"ContainerStarted","Data":"3b6c89273f06106257e2c856c47272a721f45c6d82ca5d088974a9429d0d6368"}
Jul 24 17:44:34.877421 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:34.877366    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f979b4bcf-wvrcq" podStartSLOduration=0.877349556 podStartE2EDuration="877.349556ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:44:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:34.87667928 +0000 UTC m=+282.081371096" watchObservedRunningTime="2026-07-24 17:44:34.877349556 +0000 UTC m=+282.082041354"
Jul 24 17:44:44.632193 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:44.632157    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:44.632586 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:44.632211    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:44.636990 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:44.636960    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:44.903087 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:44.903014    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:44:44.950534 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:44.950505    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d64b7c69c-cx46h"]
Jul 24 17:44:50.136289 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.136251    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"]
Jul 24 17:44:50.138510 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.138494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.141146 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.141123    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 17:44:50.141267 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.141169    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qz5v7\""
Jul 24 17:44:50.142182 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.142165    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 17:44:50.144306 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.144283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.144394 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.144328    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.144394 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.144352    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2th87\" (UniqueName: \"kubernetes.io/projected/2b91aacd-a87e-48e6-ae63-37e97ff10513-kube-api-access-2th87\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.149010 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.148991    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"]
Jul 24 17:44:50.245618 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.245593    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.245728 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.245631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.245728 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.245660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2th87\" (UniqueName: \"kubernetes.io/projected/2b91aacd-a87e-48e6-ae63-37e97ff10513-kube-api-access-2th87\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.246062 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.246041    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.246104 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.246042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.254427 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.254406    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2th87\" (UniqueName: \"kubernetes.io/projected/2b91aacd-a87e-48e6-ae63-37e97ff10513-kube-api-access-2th87\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.447614 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.447554    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:44:50.571402 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.571382    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"]
Jul 24 17:44:50.572958 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:44:50.572932    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b91aacd_a87e_48e6_ae63_37e97ff10513.slice/crio-339e8a0b80ac577674eda92133d7f020d649aba94dad759a4cae8c1d791536b2 WatchSource:0}: Error finding container 339e8a0b80ac577674eda92133d7f020d649aba94dad759a4cae8c1d791536b2: Status 404 returned error can't find the container with id 339e8a0b80ac577674eda92133d7f020d649aba94dad759a4cae8c1d791536b2
Jul 24 17:44:50.923475 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:50.923438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc" event={"ID":"2b91aacd-a87e-48e6-ae63-37e97ff10513","Type":"ContainerStarted","Data":"339e8a0b80ac577674eda92133d7f020d649aba94dad759a4cae8c1d791536b2"}
Jul 24 17:44:53.195247 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:53.195221    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:44:53.198687 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:53.198662    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:44:53.211517 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:53.211483    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:44:53.214808 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:53.214791    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 17:44:53.215097 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:53.215076    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:44:57.953454 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:57.953413    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerID="c0bd424116769d9ae7812ee34afff52c87cd50f6ec81a75d14a02dd34e2461cb" exitCode=0
Jul 24 17:44:57.953983 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:57.953499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc" event={"ID":"2b91aacd-a87e-48e6-ae63-37e97ff10513","Type":"ContainerDied","Data":"c0bd424116769d9ae7812ee34afff52c87cd50f6ec81a75d14a02dd34e2461cb"}
Jul 24 17:44:57.954600 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:44:57.954584    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:45:00.967915 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:00.967862    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerID="8389767a39e1cddfc30e732e00217a4bd51b7f3003333b510622a79e69034cfd" exitCode=0
Jul 24 17:45:00.968295 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:00.967947    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc" event={"ID":"2b91aacd-a87e-48e6-ae63-37e97ff10513","Type":"ContainerDied","Data":"8389767a39e1cddfc30e732e00217a4bd51b7f3003333b510622a79e69034cfd"}
Jul 24 17:45:09.005138 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:09.005073    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerID="e9ab51f12bfb5b73c8e5ed057b53464b58702b4a5bd15474b7f4c69045e07a76" exitCode=0
Jul 24 17:45:09.005463 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:09.005150    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc" event={"ID":"2b91aacd-a87e-48e6-ae63-37e97ff10513","Type":"ContainerDied","Data":"e9ab51f12bfb5b73c8e5ed057b53464b58702b4a5bd15474b7f4c69045e07a76"}
Jul 24 17:45:09.976179 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:09.976127    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d64b7c69c-cx46h" podUID="bb1e21a6-5747-432c-809c-f614ebd8e211" containerName="console" containerID="cri-o://810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31" gracePeriod=15
Jul 24 17:45:10.141892 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.141854    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:45:10.203279 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.203255    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-util\") pod \"2b91aacd-a87e-48e6-ae63-37e97ff10513\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") "
Jul 24 17:45:10.203416 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.203310    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-bundle\") pod \"2b91aacd-a87e-48e6-ae63-37e97ff10513\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") "
Jul 24 17:45:10.203416 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.203345    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2th87\" (UniqueName: \"kubernetes.io/projected/2b91aacd-a87e-48e6-ae63-37e97ff10513-kube-api-access-2th87\") pod \"2b91aacd-a87e-48e6-ae63-37e97ff10513\" (UID: \"2b91aacd-a87e-48e6-ae63-37e97ff10513\") "
Jul 24 17:45:10.203979 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.203958    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-bundle" (OuterVolumeSpecName: "bundle") pod "2b91aacd-a87e-48e6-ae63-37e97ff10513" (UID: "2b91aacd-a87e-48e6-ae63-37e97ff10513"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:45:10.205551 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.205522    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b91aacd-a87e-48e6-ae63-37e97ff10513-kube-api-access-2th87" (OuterVolumeSpecName: "kube-api-access-2th87") pod "2b91aacd-a87e-48e6-ae63-37e97ff10513" (UID: "2b91aacd-a87e-48e6-ae63-37e97ff10513"). InnerVolumeSpecName "kube-api-access-2th87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:45:10.208113 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.208087    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-util" (OuterVolumeSpecName: "util") pod "2b91aacd-a87e-48e6-ae63-37e97ff10513" (UID: "2b91aacd-a87e-48e6-ae63-37e97ff10513"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:45:10.219344 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.219320    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:45:10.303848 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.303825    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-serving-cert\") pod \"bb1e21a6-5747-432c-809c-f614ebd8e211\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") "
Jul 24 17:45:10.303965 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.303868    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdfn7\" (UniqueName: \"kubernetes.io/projected/bb1e21a6-5747-432c-809c-f614ebd8e211-kube-api-access-xdfn7\") pod \"bb1e21a6-5747-432c-809c-f614ebd8e211\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") "
Jul 24 17:45:10.304040 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.303998    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-console-config\") pod \"bb1e21a6-5747-432c-809c-f614ebd8e211\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") "
Jul 24 17:45:10.304097 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304056    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-oauth-serving-cert\") pod \"bb1e21a6-5747-432c-809c-f614ebd8e211\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") "
Jul 24 17:45:10.304097 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304091    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-oauth-config\") pod \"bb1e21a6-5747-432c-809c-f614ebd8e211\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") "
Jul 24 17:45:10.304202 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304136    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-trusted-ca-bundle\") pod \"bb1e21a6-5747-432c-809c-f614ebd8e211\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") "
Jul 24 17:45:10.304202 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304179    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-service-ca\") pod \"bb1e21a6-5747-432c-809c-f614ebd8e211\" (UID: \"bb1e21a6-5747-432c-809c-f614ebd8e211\") "
Jul 24 17:45:10.304404 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304382    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-console-config" (OuterVolumeSpecName: "console-config") pod "bb1e21a6-5747-432c-809c-f614ebd8e211" (UID: "bb1e21a6-5747-432c-809c-f614ebd8e211"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:45:10.304404 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304395    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:10.304578 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304412    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2th87\" (UniqueName: \"kubernetes.io/projected/2b91aacd-a87e-48e6-ae63-37e97ff10513-kube-api-access-2th87\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:10.304578 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304426    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b91aacd-a87e-48e6-ae63-37e97ff10513-util\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:10.304578 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304421    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bb1e21a6-5747-432c-809c-f614ebd8e211" (UID: "bb1e21a6-5747-432c-809c-f614ebd8e211"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:45:10.304728 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304679    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bb1e21a6-5747-432c-809c-f614ebd8e211" (UID: "bb1e21a6-5747-432c-809c-f614ebd8e211"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:45:10.304728 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.304714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-service-ca" (OuterVolumeSpecName: "service-ca") pod "bb1e21a6-5747-432c-809c-f614ebd8e211" (UID: "bb1e21a6-5747-432c-809c-f614ebd8e211"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:45:10.305940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.305917    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bb1e21a6-5747-432c-809c-f614ebd8e211" (UID: "bb1e21a6-5747-432c-809c-f614ebd8e211"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:45:10.306067 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.306047    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1e21a6-5747-432c-809c-f614ebd8e211-kube-api-access-xdfn7" (OuterVolumeSpecName: "kube-api-access-xdfn7") pod "bb1e21a6-5747-432c-809c-f614ebd8e211" (UID: "bb1e21a6-5747-432c-809c-f614ebd8e211"). InnerVolumeSpecName "kube-api-access-xdfn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:45:10.306138 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.306119    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bb1e21a6-5747-432c-809c-f614ebd8e211" (UID: "bb1e21a6-5747-432c-809c-f614ebd8e211"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:45:10.405168 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.405135    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-oauth-serving-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:10.405168 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.405165    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-oauth-config\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:10.405168 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.405175    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-trusted-ca-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:10.405317 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.405183    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:10.405317 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.405192    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb1e21a6-5747-432c-809c-f614ebd8e211-console-serving-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:10.405317 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.405200    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdfn7\" (UniqueName: \"kubernetes.io/projected/bb1e21a6-5747-432c-809c-f614ebd8e211-kube-api-access-xdfn7\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:10.405317 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:10.405208    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb1e21a6-5747-432c-809c-f614ebd8e211-console-config\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:11.014430 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.014402    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc"
Jul 24 17:45:11.014564 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.014392    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rpfhc" event={"ID":"2b91aacd-a87e-48e6-ae63-37e97ff10513","Type":"ContainerDied","Data":"339e8a0b80ac577674eda92133d7f020d649aba94dad759a4cae8c1d791536b2"}
Jul 24 17:45:11.014564 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.014531    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="339e8a0b80ac577674eda92133d7f020d649aba94dad759a4cae8c1d791536b2"
Jul 24 17:45:11.015561 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.015541    2562 generic.go:356] "Generic (PLEG): container finished" podID="bb1e21a6-5747-432c-809c-f614ebd8e211" containerID="810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31" exitCode=0
Jul 24 17:45:11.015667 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.015579    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d64b7c69c-cx46h"
Jul 24 17:45:11.015667 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.015593    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d64b7c69c-cx46h" event={"ID":"bb1e21a6-5747-432c-809c-f614ebd8e211","Type":"ContainerDied","Data":"810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31"}
Jul 24 17:45:11.015667 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.015623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d64b7c69c-cx46h" event={"ID":"bb1e21a6-5747-432c-809c-f614ebd8e211","Type":"ContainerDied","Data":"ae4b0e31f5876a22318198b581b432788e55120dd51a8b314c444a63bd22f2d7"}
Jul 24 17:45:11.015667 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.015643    2562 scope.go:117] "RemoveContainer" containerID="810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31"
Jul 24 17:45:11.024234 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.024218    2562 scope.go:117] "RemoveContainer" containerID="810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31"
Jul 24 17:45:11.024515 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:45:11.024488    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31\": container with ID starting with 810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31 not found: ID does not exist" containerID="810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31"
Jul 24 17:45:11.024628 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.024523    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31"} err="failed to get container status \"810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31\": rpc error: code = NotFound desc = could not find container \"810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31\": container with ID starting with 810351e70187f4a0f9d133e4e42a545f549ded183a26fcb28612736162075d31 not found: ID does not exist"
Jul 24 17:45:11.041855 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.041830    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d64b7c69c-cx46h"]
Jul 24 17:45:11.049991 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.045711    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d64b7c69c-cx46h"]
Jul 24 17:45:11.405919 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:11.405893    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb1e21a6-5747-432c-809c-f614ebd8e211" path="/var/lib/kubelet/pods/bb1e21a6-5747-432c-809c-f614ebd8e211/volumes"
Jul 24 17:45:18.477496 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477457    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"]
Jul 24 17:45:18.477908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477777    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerName="pull"
Jul 24 17:45:18.477908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477789    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerName="pull"
Jul 24 17:45:18.477908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477800    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerName="extract"
Jul 24 17:45:18.477908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477807    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerName="extract"
Jul 24 17:45:18.477908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477816    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb1e21a6-5747-432c-809c-f614ebd8e211" containerName="console"
Jul 24 17:45:18.477908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477822    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1e21a6-5747-432c-809c-f614ebd8e211" containerName="console"
Jul 24 17:45:18.477908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477834    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerName="util"
Jul 24 17:45:18.477908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477840    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerName="util"
Jul 24 17:45:18.477908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477911    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b91aacd-a87e-48e6-ae63-37e97ff10513" containerName="extract"
Jul 24 17:45:18.478198 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.477924    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb1e21a6-5747-432c-809c-f614ebd8e211" containerName="console"
Jul 24 17:45:18.479808 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.479787    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"
Jul 24 17:45:18.483149 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.483121    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:45:18.483460 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.483441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 24 17:45:18.483532 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.483439    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ts78d\""
Jul 24 17:45:18.496728 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.496701    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"]
Jul 24 17:45:18.559971 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.559944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trxpf\" (UniqueName: \"kubernetes.io/projected/c37fea9d-8874-410a-b765-aa29734bde9c-kube-api-access-trxpf\") pod \"cert-manager-operator-controller-manager-55c444bf86-rz8dl\" (UID: \"c37fea9d-8874-410a-b765-aa29734bde9c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"
Jul 24 17:45:18.560083 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.560003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c37fea9d-8874-410a-b765-aa29734bde9c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rz8dl\" (UID: \"c37fea9d-8874-410a-b765-aa29734bde9c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"
Jul 24 17:45:18.661174 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.661147    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c37fea9d-8874-410a-b765-aa29734bde9c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rz8dl\" (UID: \"c37fea9d-8874-410a-b765-aa29734bde9c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"
Jul 24 17:45:18.661258 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.661210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trxpf\" (UniqueName: \"kubernetes.io/projected/c37fea9d-8874-410a-b765-aa29734bde9c-kube-api-access-trxpf\") pod \"cert-manager-operator-controller-manager-55c444bf86-rz8dl\" (UID: \"c37fea9d-8874-410a-b765-aa29734bde9c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"
Jul 24 17:45:18.661607 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.661587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c37fea9d-8874-410a-b765-aa29734bde9c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rz8dl\" (UID: \"c37fea9d-8874-410a-b765-aa29734bde9c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"
Jul 24 17:45:18.670690 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.670658    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trxpf\" (UniqueName: \"kubernetes.io/projected/c37fea9d-8874-410a-b765-aa29734bde9c-kube-api-access-trxpf\") pod \"cert-manager-operator-controller-manager-55c444bf86-rz8dl\" (UID: \"c37fea9d-8874-410a-b765-aa29734bde9c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"
Jul 24 17:45:18.790031 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.790011    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"
Jul 24 17:45:18.925260 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:18.925234    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl"]
Jul 24 17:45:18.933213 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:45:18.933185    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc37fea9d_8874_410a_b765_aa29734bde9c.slice/crio-4a2f39d8d80e02fb22e7050fd4466cf17542e05e1b0306ca57e2aa27f7cd5d48 WatchSource:0}: Error finding container 4a2f39d8d80e02fb22e7050fd4466cf17542e05e1b0306ca57e2aa27f7cd5d48: Status 404 returned error can't find the container with id 4a2f39d8d80e02fb22e7050fd4466cf17542e05e1b0306ca57e2aa27f7cd5d48
Jul 24 17:45:19.049493 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:19.049426    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl" event={"ID":"c37fea9d-8874-410a-b765-aa29734bde9c","Type":"ContainerStarted","Data":"4a2f39d8d80e02fb22e7050fd4466cf17542e05e1b0306ca57e2aa27f7cd5d48"}
Jul 24 17:45:21.059608 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:21.059572    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl" event={"ID":"c37fea9d-8874-410a-b765-aa29734bde9c","Type":"ContainerStarted","Data":"721b620900fc112525942409c60b8e126c392f5d1afe0547edf8f88628b09fb2"}
Jul 24 17:45:21.089929 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:21.089888    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rz8dl" podStartSLOduration=1.240657685 podStartE2EDuration="3.089858978s" totalImagesPullingTime="1.849201293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:21.088004642 +0000 UTC m=+328.292696441" watchObservedRunningTime="2026-07-24 17:45:21.089858978 +0000 UTC m=+328.294550778"
Jul 24 17:45:22.174264 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.174229    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"]
Jul 24 17:45:22.176719 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.176702    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.179551 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.179530    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 17:45:22.180641 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.180627    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qz5v7\""
Jul 24 17:45:22.180697 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.180640    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 17:45:22.187270 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.187248    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"]
Jul 24 17:45:22.288010 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.287983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtr4k\" (UniqueName: \"kubernetes.io/projected/98c0da57-d388-4e36-bcaa-18f054e2cdfe-kube-api-access-jtr4k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.288109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.288022    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.288109 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.288094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.389303 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.389275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.389407 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.389310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtr4k\" (UniqueName: \"kubernetes.io/projected/98c0da57-d388-4e36-bcaa-18f054e2cdfe-kube-api-access-jtr4k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.389407 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.389342    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.389652 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.389632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.389692 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.389671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.398752 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.398727    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtr4k\" (UniqueName: \"kubernetes.io/projected/98c0da57-d388-4e36-bcaa-18f054e2cdfe-kube-api-access-jtr4k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.485453 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.485389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:22.610587 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:22.610520    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"]
Jul 24 17:45:22.612742 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:45:22.612707    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c0da57_d388_4e36_bcaa_18f054e2cdfe.slice/crio-53548a8169850689933c5a87444e5b96ff9ab248223da0c3ddcf983eb8ae297b WatchSource:0}: Error finding container 53548a8169850689933c5a87444e5b96ff9ab248223da0c3ddcf983eb8ae297b: Status 404 returned error can't find the container with id 53548a8169850689933c5a87444e5b96ff9ab248223da0c3ddcf983eb8ae297b
Jul 24 17:45:23.069180 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:23.069144    2562 generic.go:356] "Generic (PLEG): container finished" podID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerID="2ab39a7b464fa0c438098b6c0e57f75584acd0e027c126aff29c1d230dff7d93" exitCode=0
Jul 24 17:45:23.069325 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:23.069241    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q" event={"ID":"98c0da57-d388-4e36-bcaa-18f054e2cdfe","Type":"ContainerDied","Data":"2ab39a7b464fa0c438098b6c0e57f75584acd0e027c126aff29c1d230dff7d93"}
Jul 24 17:45:23.069325 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:23.069298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q" event={"ID":"98c0da57-d388-4e36-bcaa-18f054e2cdfe","Type":"ContainerStarted","Data":"53548a8169850689933c5a87444e5b96ff9ab248223da0c3ddcf983eb8ae297b"}
Jul 24 17:45:25.079807 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:25.079781    2562 generic.go:356] "Generic (PLEG): container finished" podID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerID="bb115d77cffe069be3ee6f160673b80e45c33aab91d9bb0bcc3b15d28c6d73ae" exitCode=0
Jul 24 17:45:25.080103 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:25.079863    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q" event={"ID":"98c0da57-d388-4e36-bcaa-18f054e2cdfe","Type":"ContainerDied","Data":"bb115d77cffe069be3ee6f160673b80e45c33aab91d9bb0bcc3b15d28c6d73ae"}
Jul 24 17:45:26.086444 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:26.086413    2562 generic.go:356] "Generic (PLEG): container finished" podID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerID="2527d8f01cb2afbd003a8cd2c279e9e76bd47b4f45f67ebd9476068f0ce9a006" exitCode=0
Jul 24 17:45:26.086808 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:26.086455    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q" event={"ID":"98c0da57-d388-4e36-bcaa-18f054e2cdfe","Type":"ContainerDied","Data":"2527d8f01cb2afbd003a8cd2c279e9e76bd47b4f45f67ebd9476068f0ce9a006"}
Jul 24 17:45:27.220613 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.220592    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:27.327470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.327442    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-bundle\") pod \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") "
Jul 24 17:45:27.327607 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.327486    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-util\") pod \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") "
Jul 24 17:45:27.327681 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.327605    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtr4k\" (UniqueName: \"kubernetes.io/projected/98c0da57-d388-4e36-bcaa-18f054e2cdfe-kube-api-access-jtr4k\") pod \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\" (UID: \"98c0da57-d388-4e36-bcaa-18f054e2cdfe\") "
Jul 24 17:45:27.327869 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.327846    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-bundle" (OuterVolumeSpecName: "bundle") pod "98c0da57-d388-4e36-bcaa-18f054e2cdfe" (UID: "98c0da57-d388-4e36-bcaa-18f054e2cdfe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:45:27.329508 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.329484    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c0da57-d388-4e36-bcaa-18f054e2cdfe-kube-api-access-jtr4k" (OuterVolumeSpecName: "kube-api-access-jtr4k") pod "98c0da57-d388-4e36-bcaa-18f054e2cdfe" (UID: "98c0da57-d388-4e36-bcaa-18f054e2cdfe"). InnerVolumeSpecName "kube-api-access-jtr4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:45:27.333043 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.333019    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-util" (OuterVolumeSpecName: "util") pod "98c0da57-d388-4e36-bcaa-18f054e2cdfe" (UID: "98c0da57-d388-4e36-bcaa-18f054e2cdfe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:45:27.428717 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.428697    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtr4k\" (UniqueName: \"kubernetes.io/projected/98c0da57-d388-4e36-bcaa-18f054e2cdfe-kube-api-access-jtr4k\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:27.428717 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.428717    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:27.428848 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:27.428726    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98c0da57-d388-4e36-bcaa-18f054e2cdfe-util\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:28.098596 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:28.098574    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q"
Jul 24 17:45:28.098596 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:28.098588    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jv8q" event={"ID":"98c0da57-d388-4e36-bcaa-18f054e2cdfe","Type":"ContainerDied","Data":"53548a8169850689933c5a87444e5b96ff9ab248223da0c3ddcf983eb8ae297b"}
Jul 24 17:45:28.098768 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:28.098628    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53548a8169850689933c5a87444e5b96ff9ab248223da0c3ddcf983eb8ae297b"
Jul 24 17:45:33.454519 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.454482    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jvjj9"]
Jul 24 17:45:33.454943 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.454789    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerName="util"
Jul 24 17:45:33.454943 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.454802    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerName="util"
Jul 24 17:45:33.454943 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.454818    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerName="extract"
Jul 24 17:45:33.454943 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.454824    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerName="extract"
Jul 24 17:45:33.454943 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.454838    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerName="pull"
Jul 24 17:45:33.454943 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.454846    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerName="pull"
Jul 24 17:45:33.454943 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.454920    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c0da57-d388-4e36-bcaa-18f054e2cdfe" containerName="extract"
Jul 24 17:45:33.458009 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.457988    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jvjj9"
Jul 24 17:45:33.460715 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.460691    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 17:45:33.460852 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.460788    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-44wsc\""
Jul 24 17:45:33.461778 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.461758    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 17:45:33.468758 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.468731    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jvjj9"]
Jul 24 17:45:33.575015 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.574989    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092-bound-sa-token\") pod \"cert-manager-55b4d954d5-jvjj9\" (UID: \"6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092\") " pod="cert-manager/cert-manager-55b4d954d5-jvjj9"
Jul 24 17:45:33.575128 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.575021    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pqf5\" (UniqueName: \"kubernetes.io/projected/6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092-kube-api-access-5pqf5\") pod \"cert-manager-55b4d954d5-jvjj9\" (UID: \"6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092\") " pod="cert-manager/cert-manager-55b4d954d5-jvjj9"
Jul 24 17:45:33.676325 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.676296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092-bound-sa-token\") pod \"cert-manager-55b4d954d5-jvjj9\" (UID: \"6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092\") " pod="cert-manager/cert-manager-55b4d954d5-jvjj9"
Jul 24 17:45:33.676446 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.676329    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pqf5\" (UniqueName: \"kubernetes.io/projected/6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092-kube-api-access-5pqf5\") pod \"cert-manager-55b4d954d5-jvjj9\" (UID: \"6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092\") " pod="cert-manager/cert-manager-55b4d954d5-jvjj9"
Jul 24 17:45:33.685647 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.685613    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092-bound-sa-token\") pod \"cert-manager-55b4d954d5-jvjj9\" (UID: \"6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092\") " pod="cert-manager/cert-manager-55b4d954d5-jvjj9"
Jul 24 17:45:33.685766 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.685680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pqf5\" (UniqueName: \"kubernetes.io/projected/6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092-kube-api-access-5pqf5\") pod \"cert-manager-55b4d954d5-jvjj9\" (UID: \"6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092\") " pod="cert-manager/cert-manager-55b4d954d5-jvjj9"
Jul 24 17:45:33.769770 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.769715    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jvjj9"
Jul 24 17:45:33.895986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:33.895957    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jvjj9"]
Jul 24 17:45:33.898213 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:45:33.898186    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fe0af01_8bfe_4dd8_b8fb_3d59f48d8092.slice/crio-020ad55bd8997b8c6e3339401f8ad4773ef5ec7503062db8db35e9e42e830ae2 WatchSource:0}: Error finding container 020ad55bd8997b8c6e3339401f8ad4773ef5ec7503062db8db35e9e42e830ae2: Status 404 returned error can't find the container with id 020ad55bd8997b8c6e3339401f8ad4773ef5ec7503062db8db35e9e42e830ae2
Jul 24 17:45:34.126020 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.125987    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jvjj9" event={"ID":"6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092","Type":"ContainerStarted","Data":"020ad55bd8997b8c6e3339401f8ad4773ef5ec7503062db8db35e9e42e830ae2"}
Jul 24 17:45:34.735065 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.735028    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"]
Jul 24 17:45:34.738340 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.738319    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"
Jul 24 17:45:34.740947 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.740928    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 17:45:34.741945 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.741922    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-hjggz\""
Jul 24 17:45:34.742054 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.741922    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:45:34.748186 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.747626    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"]
Jul 24 17:45:34.885151 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.885114    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grscw\" (UniqueName: \"kubernetes.io/projected/4ed36f70-2113-40ee-a001-5f52a07f7cdc-kube-api-access-grscw\") pod \"openshift-lws-operator-fd8ccff4c-ld5t2\" (UID: \"4ed36f70-2113-40ee-a001-5f52a07f7cdc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"
Jul 24 17:45:34.885298 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.885243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ed36f70-2113-40ee-a001-5f52a07f7cdc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ld5t2\" (UID: \"4ed36f70-2113-40ee-a001-5f52a07f7cdc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"
Jul 24 17:45:34.986571 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.986504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ed36f70-2113-40ee-a001-5f52a07f7cdc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ld5t2\" (UID: \"4ed36f70-2113-40ee-a001-5f52a07f7cdc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"
Jul 24 17:45:34.986571 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.986556    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grscw\" (UniqueName: \"kubernetes.io/projected/4ed36f70-2113-40ee-a001-5f52a07f7cdc-kube-api-access-grscw\") pod \"openshift-lws-operator-fd8ccff4c-ld5t2\" (UID: \"4ed36f70-2113-40ee-a001-5f52a07f7cdc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"
Jul 24 17:45:34.986904 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.986868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ed36f70-2113-40ee-a001-5f52a07f7cdc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ld5t2\" (UID: \"4ed36f70-2113-40ee-a001-5f52a07f7cdc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"
Jul 24 17:45:34.995631 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:34.995605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grscw\" (UniqueName: \"kubernetes.io/projected/4ed36f70-2113-40ee-a001-5f52a07f7cdc-kube-api-access-grscw\") pod \"openshift-lws-operator-fd8ccff4c-ld5t2\" (UID: \"4ed36f70-2113-40ee-a001-5f52a07f7cdc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"
Jul 24 17:45:35.051140 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:35.051114    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"
Jul 24 17:45:35.198500 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:35.198356    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2"]
Jul 24 17:45:35.200771 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:45:35.200745    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ed36f70_2113_40ee_a001_5f52a07f7cdc.slice/crio-2647c881e2ae241ee9823bf4e22363483463e078220f4dda11fea979cf4cf790 WatchSource:0}: Error finding container 2647c881e2ae241ee9823bf4e22363483463e078220f4dda11fea979cf4cf790: Status 404 returned error can't find the container with id 2647c881e2ae241ee9823bf4e22363483463e078220f4dda11fea979cf4cf790
Jul 24 17:45:36.138248 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:36.138205    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2" event={"ID":"4ed36f70-2113-40ee-a001-5f52a07f7cdc","Type":"ContainerStarted","Data":"2647c881e2ae241ee9823bf4e22363483463e078220f4dda11fea979cf4cf790"}
Jul 24 17:45:37.145455 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:37.145408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jvjj9" event={"ID":"6fe0af01-8bfe-4dd8-b8fb-3d59f48d8092","Type":"ContainerStarted","Data":"9b3a2e1e23337d657ea813dbb7cc7aab8594bcf5d430cf32e35e4b839b052ac3"}
Jul 24 17:45:37.165522 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:37.165470    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-jvjj9" podStartSLOduration=1.6717567770000001 podStartE2EDuration="4.165455645s" totalImagesPullingTime="2.493698868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:37.164070603 +0000 UTC m=+344.368762400" watchObservedRunningTime="2026-07-24 17:45:37.165455645 +0000 UTC m=+344.370147441"
Jul 24 17:45:38.152190 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:38.152152    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2" event={"ID":"4ed36f70-2113-40ee-a001-5f52a07f7cdc","Type":"ContainerStarted","Data":"9fbde54fc32836f6dda39cdd4a70f63b2c1ce72f9b44edb6b2ccb9e5005901c6"}
Jul 24 17:45:38.175573 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:38.175526    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ld5t2" podStartSLOduration=1.761245575 podStartE2EDuration="4.175510411s" totalImagesPullingTime="2.414264836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:38.174275483 +0000 UTC m=+345.378967290" watchObservedRunningTime="2026-07-24 17:45:38.175510411 +0000 UTC m=+345.380202208"
Jul 24 17:45:42.573709 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.573667    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"]
Jul 24 17:45:42.595611 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.595580    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"]
Jul 24 17:45:42.595766 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.595690    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.598512 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.598487    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 17:45:42.598659 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.598489    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qz5v7\""
Jul 24 17:45:42.599456 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.599436    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 17:45:42.746843 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.746820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcp7k\" (UniqueName: \"kubernetes.io/projected/6338f78c-852c-45bf-b7db-1740207aca2d-kube-api-access-rcp7k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.746973 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.746869    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.746973 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.746954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.847660 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.847601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.847660 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.847638    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.847792 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.847667    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcp7k\" (UniqueName: \"kubernetes.io/projected/6338f78c-852c-45bf-b7db-1740207aca2d-kube-api-access-rcp7k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.848052 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.848030    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.848052 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.848043    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.856231 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.856194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcp7k\" (UniqueName: \"kubernetes.io/projected/6338f78c-852c-45bf-b7db-1740207aca2d-kube-api-access-rcp7k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:42.905843 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:42.905818    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:43.049429 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:43.049398    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"]
Jul 24 17:45:43.051195 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:45:43.051171    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6338f78c_852c_45bf_b7db_1740207aca2d.slice/crio-4a290cc636aa6bf5e436b9c34e7a6ab78b47a0840e41a211541ffcc9732818df WatchSource:0}: Error finding container 4a290cc636aa6bf5e436b9c34e7a6ab78b47a0840e41a211541ffcc9732818df: Status 404 returned error can't find the container with id 4a290cc636aa6bf5e436b9c34e7a6ab78b47a0840e41a211541ffcc9732818df
Jul 24 17:45:43.175104 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:43.175074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz" event={"ID":"6338f78c-852c-45bf-b7db-1740207aca2d","Type":"ContainerStarted","Data":"cf3fa7e3a024bc9cbecb7ff0bff1d3dce72520556088130dacf9ff2f29d1c92c"}
Jul 24 17:45:43.175257 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:43.175127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz" event={"ID":"6338f78c-852c-45bf-b7db-1740207aca2d","Type":"ContainerStarted","Data":"4a290cc636aa6bf5e436b9c34e7a6ab78b47a0840e41a211541ffcc9732818df"}
Jul 24 17:45:44.181509 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:44.181435    2562 generic.go:356] "Generic (PLEG): container finished" podID="6338f78c-852c-45bf-b7db-1740207aca2d" containerID="cf3fa7e3a024bc9cbecb7ff0bff1d3dce72520556088130dacf9ff2f29d1c92c" exitCode=0
Jul 24 17:45:44.181509 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:44.181477    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz" event={"ID":"6338f78c-852c-45bf-b7db-1740207aca2d","Type":"ContainerDied","Data":"cf3fa7e3a024bc9cbecb7ff0bff1d3dce72520556088130dacf9ff2f29d1c92c"}
Jul 24 17:45:45.188415 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:45.188384    2562 generic.go:356] "Generic (PLEG): container finished" podID="6338f78c-852c-45bf-b7db-1740207aca2d" containerID="c63cad9e2ddd8005cb949afac9be4ea9334df96abe16548abf133fc6259c7e20" exitCode=0
Jul 24 17:45:45.188737 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:45.188498    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz" event={"ID":"6338f78c-852c-45bf-b7db-1740207aca2d","Type":"ContainerDied","Data":"c63cad9e2ddd8005cb949afac9be4ea9334df96abe16548abf133fc6259c7e20"}
Jul 24 17:45:46.195435 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:46.195393    2562 generic.go:356] "Generic (PLEG): container finished" podID="6338f78c-852c-45bf-b7db-1740207aca2d" containerID="111a94fa80fa359fa7c6c08dea43e7e6d12e2ebd97bb25ab6239961c8263799c" exitCode=0
Jul 24 17:45:46.195858 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:46.195474    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz" event={"ID":"6338f78c-852c-45bf-b7db-1740207aca2d","Type":"ContainerDied","Data":"111a94fa80fa359fa7c6c08dea43e7e6d12e2ebd97bb25ab6239961c8263799c"}
Jul 24 17:45:47.320646 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.320621    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:47.482830 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.482767    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcp7k\" (UniqueName: \"kubernetes.io/projected/6338f78c-852c-45bf-b7db-1740207aca2d-kube-api-access-rcp7k\") pod \"6338f78c-852c-45bf-b7db-1740207aca2d\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") "
Jul 24 17:45:47.482830 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.482800    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-bundle\") pod \"6338f78c-852c-45bf-b7db-1740207aca2d\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") "
Jul 24 17:45:47.483052 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.482850    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-util\") pod \"6338f78c-852c-45bf-b7db-1740207aca2d\" (UID: \"6338f78c-852c-45bf-b7db-1740207aca2d\") "
Jul 24 17:45:47.483601 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.483575    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-bundle" (OuterVolumeSpecName: "bundle") pod "6338f78c-852c-45bf-b7db-1740207aca2d" (UID: "6338f78c-852c-45bf-b7db-1740207aca2d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:45:47.484938 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.484906    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6338f78c-852c-45bf-b7db-1740207aca2d-kube-api-access-rcp7k" (OuterVolumeSpecName: "kube-api-access-rcp7k") pod "6338f78c-852c-45bf-b7db-1740207aca2d" (UID: "6338f78c-852c-45bf-b7db-1740207aca2d"). InnerVolumeSpecName "kube-api-access-rcp7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:45:47.488612 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.488590    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-util" (OuterVolumeSpecName: "util") pod "6338f78c-852c-45bf-b7db-1740207aca2d" (UID: "6338f78c-852c-45bf-b7db-1740207aca2d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:45:47.583408 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.583385    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-util\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:47.583408 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.583407    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcp7k\" (UniqueName: \"kubernetes.io/projected/6338f78c-852c-45bf-b7db-1740207aca2d-kube-api-access-rcp7k\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:47.583533 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:47.583418    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6338f78c-852c-45bf-b7db-1740207aca2d-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:45:48.206479 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:48.206446    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz"
Jul 24 17:45:48.206479 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:48.206458    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwcqz" event={"ID":"6338f78c-852c-45bf-b7db-1740207aca2d","Type":"ContainerDied","Data":"4a290cc636aa6bf5e436b9c34e7a6ab78b47a0840e41a211541ffcc9732818df"}
Jul 24 17:45:48.206674 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:48.206519    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a290cc636aa6bf5e436b9c34e7a6ab78b47a0840e41a211541ffcc9732818df"
Jul 24 17:45:54.876896 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.876853    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"]
Jul 24 17:45:54.877360 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.877280    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6338f78c-852c-45bf-b7db-1740207aca2d" containerName="pull"
Jul 24 17:45:54.877360 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.877298    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6338f78c-852c-45bf-b7db-1740207aca2d" containerName="pull"
Jul 24 17:45:54.877360 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.877309    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6338f78c-852c-45bf-b7db-1740207aca2d" containerName="extract"
Jul 24 17:45:54.877360 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.877318    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6338f78c-852c-45bf-b7db-1740207aca2d" containerName="extract"
Jul 24 17:45:54.877360 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.877356    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6338f78c-852c-45bf-b7db-1740207aca2d" containerName="util"
Jul 24 17:45:54.877620 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.877364    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6338f78c-852c-45bf-b7db-1740207aca2d" containerName="util"
Jul 24 17:45:54.877620 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.877446    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6338f78c-852c-45bf-b7db-1740207aca2d" containerName="extract"
Jul 24 17:45:54.883036 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.883015    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:54.886923 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.886899    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 17:45:54.887049 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.887010    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 17:45:54.887141 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.887123    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 17:45:54.887196 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.887132    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 17:45:54.887337 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.887319    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ng8ql\""
Jul 24 17:45:54.902742 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:54.902712    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"]
Jul 24 17:45:55.039530 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.039493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr9w2\" (UniqueName: \"kubernetes.io/projected/9d5925b1-46b5-4431-a612-bf3010884213-kube-api-access-qr9w2\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.039708 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.039540    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9d5925b1-46b5-4431-a612-bf3010884213-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.039708 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.039604    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d5925b1-46b5-4431-a612-bf3010884213-tmp\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.039708 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.039668    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9d5925b1-46b5-4431-a612-bf3010884213-manifests\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.039825 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.039743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d5925b1-46b5-4431-a612-bf3010884213-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.141114 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.141037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9d5925b1-46b5-4431-a612-bf3010884213-manifests\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.141114 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.141101    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d5925b1-46b5-4431-a612-bf3010884213-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.141309 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.141130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr9w2\" (UniqueName: \"kubernetes.io/projected/9d5925b1-46b5-4431-a612-bf3010884213-kube-api-access-qr9w2\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.141309 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.141153    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9d5925b1-46b5-4431-a612-bf3010884213-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.141309 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.141171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d5925b1-46b5-4431-a612-bf3010884213-tmp\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.141524 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.141498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9d5925b1-46b5-4431-a612-bf3010884213-manifests\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.143542 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.143518    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d5925b1-46b5-4431-a612-bf3010884213-tmp\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.143741 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.143722    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d5925b1-46b5-4431-a612-bf3010884213-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.143800 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.143742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9d5925b1-46b5-4431-a612-bf3010884213-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.149442 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.149415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr9w2\" (UniqueName: \"kubernetes.io/projected/9d5925b1-46b5-4431-a612-bf3010884213-kube-api-access-qr9w2\") pod \"opendatahub-operator-controller-manager-67f579866-ftf6f\" (UID: \"9d5925b1-46b5-4431-a612-bf3010884213\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.193546 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.193521    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:45:55.351278 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:55.351188    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"]
Jul 24 17:45:55.353480 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:45:55.353453    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d5925b1_46b5_4431_a612_bf3010884213.slice/crio-5bb2b45754e80dc24275bec6ad196e34aa9f9be0f91f9b62b3db3ae11bf7943b WatchSource:0}: Error finding container 5bb2b45754e80dc24275bec6ad196e34aa9f9be0f91f9b62b3db3ae11bf7943b: Status 404 returned error can't find the container with id 5bb2b45754e80dc24275bec6ad196e34aa9f9be0f91f9b62b3db3ae11bf7943b
Jul 24 17:45:56.248528 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:56.248490    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f" event={"ID":"9d5925b1-46b5-4431-a612-bf3010884213","Type":"ContainerStarted","Data":"5bb2b45754e80dc24275bec6ad196e34aa9f9be0f91f9b62b3db3ae11bf7943b"}
Jul 24 17:45:59.265361 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:45:59.265317    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f" event={"ID":"9d5925b1-46b5-4431-a612-bf3010884213","Type":"ContainerStarted","Data":"953844954b4820f4f6049d0b59d1ff3334a0b306abe6a35ed6b8c8bfca4f5833"}
Jul 24 17:46:00.271070 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:00.271038    2562 generic.go:356] "Generic (PLEG): container finished" podID="9d5925b1-46b5-4431-a612-bf3010884213" containerID="953844954b4820f4f6049d0b59d1ff3334a0b306abe6a35ed6b8c8bfca4f5833" exitCode=0
Jul 24 17:46:00.271432 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:00.271117    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f" event={"ID":"9d5925b1-46b5-4431-a612-bf3010884213","Type":"ContainerDied","Data":"953844954b4820f4f6049d0b59d1ff3334a0b306abe6a35ed6b8c8bfca4f5833"}
Jul 24 17:46:01.276782 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:01.276749    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f" event={"ID":"9d5925b1-46b5-4431-a612-bf3010884213","Type":"ContainerStarted","Data":"a962863cfe78d30a8ac6dc57a6cacecc4da21bf5d595d70252ab3d0d355d355e"}
Jul 24 17:46:01.277166 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:01.276852    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:46:01.300368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:01.300324    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f" podStartSLOduration=2.936830579 podStartE2EDuration="7.3003102s" totalImagesPullingTime="3.283283909s" totalInitContainerRuntime="1.080195712s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:01.298078015 +0000 UTC m=+368.502769811" watchObservedRunningTime="2026-07-24 17:46:01.3003102 +0000 UTC m=+368.505001998"
Jul 24 17:46:12.283864 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:12.283835    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-ftf6f"
Jul 24 17:46:14.677789 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.677749    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"]
Jul 24 17:46:14.685797 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.685774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.688456 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.688435    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 17:46:14.688570 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.688472    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 17:46:14.689556 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.689536    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9rqt8\""
Jul 24 17:46:14.695202 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.695178    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"]
Jul 24 17:46:14.791862 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.791836    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d58dc331-b5f6-4522-82fd-c56eaa0f150e-tls-certs\") pod \"kube-auth-proxy-69ddb4868f-2bngz\" (UID: \"d58dc331-b5f6-4522-82fd-c56eaa0f150e\") " pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.791982 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.791870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bn6m\" (UniqueName: \"kubernetes.io/projected/d58dc331-b5f6-4522-82fd-c56eaa0f150e-kube-api-access-7bn6m\") pod \"kube-auth-proxy-69ddb4868f-2bngz\" (UID: \"d58dc331-b5f6-4522-82fd-c56eaa0f150e\") " pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.791982 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.791916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d58dc331-b5f6-4522-82fd-c56eaa0f150e-tmp\") pod \"kube-auth-proxy-69ddb4868f-2bngz\" (UID: \"d58dc331-b5f6-4522-82fd-c56eaa0f150e\") " pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.800118 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.800094    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gqrlz"]
Jul 24 17:46:14.803146 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.803132    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:14.805608 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.805589    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 24 17:46:14.805758 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.805744    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wjwkg\""
Jul 24 17:46:14.822169 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.822149    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gqrlz"]
Jul 24 17:46:14.892832 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.892808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d58dc331-b5f6-4522-82fd-c56eaa0f150e-tls-certs\") pod \"kube-auth-proxy-69ddb4868f-2bngz\" (UID: \"d58dc331-b5f6-4522-82fd-c56eaa0f150e\") " pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.892972 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.892841    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bn6m\" (UniqueName: \"kubernetes.io/projected/d58dc331-b5f6-4522-82fd-c56eaa0f150e-kube-api-access-7bn6m\") pod \"kube-auth-proxy-69ddb4868f-2bngz\" (UID: \"d58dc331-b5f6-4522-82fd-c56eaa0f150e\") " pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.892972 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.892858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d58dc331-b5f6-4522-82fd-c56eaa0f150e-tmp\") pod \"kube-auth-proxy-69ddb4868f-2bngz\" (UID: \"d58dc331-b5f6-4522-82fd-c56eaa0f150e\") " pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.892972 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.892911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2snft\" (UniqueName: \"kubernetes.io/projected/d6ec55d0-2bd3-4922-8f5e-0576651571e1-kube-api-access-2snft\") pod \"odh-model-controller-7998c67dc4-gqrlz\" (UID: \"d6ec55d0-2bd3-4922-8f5e-0576651571e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:14.892972 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.892958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6ec55d0-2bd3-4922-8f5e-0576651571e1-cert\") pod \"odh-model-controller-7998c67dc4-gqrlz\" (UID: \"d6ec55d0-2bd3-4922-8f5e-0576651571e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:14.895152 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.895124    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d58dc331-b5f6-4522-82fd-c56eaa0f150e-tmp\") pod \"kube-auth-proxy-69ddb4868f-2bngz\" (UID: \"d58dc331-b5f6-4522-82fd-c56eaa0f150e\") " pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.895338 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.895318    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d58dc331-b5f6-4522-82fd-c56eaa0f150e-tls-certs\") pod \"kube-auth-proxy-69ddb4868f-2bngz\" (UID: \"d58dc331-b5f6-4522-82fd-c56eaa0f150e\") " pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.904677 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.904655    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bn6m\" (UniqueName: \"kubernetes.io/projected/d58dc331-b5f6-4522-82fd-c56eaa0f150e-kube-api-access-7bn6m\") pod \"kube-auth-proxy-69ddb4868f-2bngz\" (UID: \"d58dc331-b5f6-4522-82fd-c56eaa0f150e\") " pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:14.993678 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.993626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2snft\" (UniqueName: \"kubernetes.io/projected/d6ec55d0-2bd3-4922-8f5e-0576651571e1-kube-api-access-2snft\") pod \"odh-model-controller-7998c67dc4-gqrlz\" (UID: \"d6ec55d0-2bd3-4922-8f5e-0576651571e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:14.993678 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.993665    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6ec55d0-2bd3-4922-8f5e-0576651571e1-cert\") pod \"odh-model-controller-7998c67dc4-gqrlz\" (UID: \"d6ec55d0-2bd3-4922-8f5e-0576651571e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:14.993813 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:46:14.993780    2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 24 17:46:14.993869 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:46:14.993858    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ec55d0-2bd3-4922-8f5e-0576651571e1-cert podName:d6ec55d0-2bd3-4922-8f5e-0576651571e1 nodeName:}" failed. No retries permitted until 2026-07-24 17:46:15.493838438 +0000 UTC m=+382.698530218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6ec55d0-2bd3-4922-8f5e-0576651571e1-cert") pod "odh-model-controller-7998c67dc4-gqrlz" (UID: "d6ec55d0-2bd3-4922-8f5e-0576651571e1") : secret "odh-model-controller-webhook-cert" not found
Jul 24 17:46:14.996627 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:14.996604    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"
Jul 24 17:46:15.007548 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.007529    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2snft\" (UniqueName: \"kubernetes.io/projected/d6ec55d0-2bd3-4922-8f5e-0576651571e1-kube-api-access-2snft\") pod \"odh-model-controller-7998c67dc4-gqrlz\" (UID: \"d6ec55d0-2bd3-4922-8f5e-0576651571e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:15.145192 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:46:15.145165    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58dc331_b5f6_4522_82fd_c56eaa0f150e.slice/crio-9b4d361464dad198e733cb208617bdc803ce062ffa58d9c8f882161ab1da2ddb WatchSource:0}: Error finding container 9b4d361464dad198e733cb208617bdc803ce062ffa58d9c8f882161ab1da2ddb: Status 404 returned error can't find the container with id 9b4d361464dad198e733cb208617bdc803ce062ffa58d9c8f882161ab1da2ddb
Jul 24 17:46:15.145835 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.145784    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz"]
Jul 24 17:46:15.343545 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.343516    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz" event={"ID":"d58dc331-b5f6-4522-82fd-c56eaa0f150e","Type":"ContainerStarted","Data":"9b4d361464dad198e733cb208617bdc803ce062ffa58d9c8f882161ab1da2ddb"}
Jul 24 17:46:15.450725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.450696    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cdhx8"]
Jul 24 17:46:15.453751 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.453734    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:15.456644 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.456623    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-pjt9c\""
Jul 24 17:46:15.456806 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.456790    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 24 17:46:15.473671 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.473648    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cdhx8"]
Jul 24 17:46:15.496898 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.496857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/935e18f3-2fcc-4d93-91fa-da3b7f510aa9-tls-certs\") pod \"model-serving-api-54d89bd5fc-cdhx8\" (UID: \"935e18f3-2fcc-4d93-91fa-da3b7f510aa9\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:15.496992 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.496938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6ec55d0-2bd3-4922-8f5e-0576651571e1-cert\") pod \"odh-model-controller-7998c67dc4-gqrlz\" (UID: \"d6ec55d0-2bd3-4922-8f5e-0576651571e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:15.496992 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.496958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wr8z\" (UniqueName: \"kubernetes.io/projected/935e18f3-2fcc-4d93-91fa-da3b7f510aa9-kube-api-access-6wr8z\") pod \"model-serving-api-54d89bd5fc-cdhx8\" (UID: \"935e18f3-2fcc-4d93-91fa-da3b7f510aa9\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:15.499411 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.499391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6ec55d0-2bd3-4922-8f5e-0576651571e1-cert\") pod \"odh-model-controller-7998c67dc4-gqrlz\" (UID: \"d6ec55d0-2bd3-4922-8f5e-0576651571e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:15.597905 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.597832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wr8z\" (UniqueName: \"kubernetes.io/projected/935e18f3-2fcc-4d93-91fa-da3b7f510aa9-kube-api-access-6wr8z\") pod \"model-serving-api-54d89bd5fc-cdhx8\" (UID: \"935e18f3-2fcc-4d93-91fa-da3b7f510aa9\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:15.597905 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.597872    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/935e18f3-2fcc-4d93-91fa-da3b7f510aa9-tls-certs\") pod \"model-serving-api-54d89bd5fc-cdhx8\" (UID: \"935e18f3-2fcc-4d93-91fa-da3b7f510aa9\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:15.598072 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:46:15.598031    2562 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 24 17:46:15.598129 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:46:15.598105    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/935e18f3-2fcc-4d93-91fa-da3b7f510aa9-tls-certs podName:935e18f3-2fcc-4d93-91fa-da3b7f510aa9 nodeName:}" failed. No retries permitted until 2026-07-24 17:46:16.098086252 +0000 UTC m=+383.302778041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/935e18f3-2fcc-4d93-91fa-da3b7f510aa9-tls-certs") pod "model-serving-api-54d89bd5fc-cdhx8" (UID: "935e18f3-2fcc-4d93-91fa-da3b7f510aa9") : secret "model-serving-api-tls" not found
Jul 24 17:46:15.615362 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.615337    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wr8z\" (UniqueName: \"kubernetes.io/projected/935e18f3-2fcc-4d93-91fa-da3b7f510aa9-kube-api-access-6wr8z\") pod \"model-serving-api-54d89bd5fc-cdhx8\" (UID: \"935e18f3-2fcc-4d93-91fa-da3b7f510aa9\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:15.712194 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.712166    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:15.850080 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.850035    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gqrlz"]
Jul 24 17:46:15.852351 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:46:15.852320    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ec55d0_2bd3_4922_8f5e_0576651571e1.slice/crio-b0499c0a248a26a5fdabfdcaca7a74072a1cdae7a20532ad3d72a1cc4691570a WatchSource:0}: Error finding container b0499c0a248a26a5fdabfdcaca7a74072a1cdae7a20532ad3d72a1cc4691570a: Status 404 returned error can't find the container with id b0499c0a248a26a5fdabfdcaca7a74072a1cdae7a20532ad3d72a1cc4691570a
Jul 24 17:46:15.993221 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.993192    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"]
Jul 24 17:46:15.996632 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.996614    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:15.999872 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.999850    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 17:46:15.999994 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.999900    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 17:46:15.999994 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.999912    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 17:46:16.000086 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:15.999854    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-54ddn\""
Jul 24 17:46:16.000166 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.000149    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 17:46:16.014071 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.014047    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"]
Jul 24 17:46:16.101775 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.101712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/935e18f3-2fcc-4d93-91fa-da3b7f510aa9-tls-certs\") pod \"model-serving-api-54d89bd5fc-cdhx8\" (UID: \"935e18f3-2fcc-4d93-91fa-da3b7f510aa9\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:16.101775 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.101755    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.101775 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.101773    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.102063 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.101808    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.102063 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.101836    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.102063 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.101863    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.102063 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.101983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.102063 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.102040    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ddh2\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-kube-api-access-8ddh2\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.104441 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.104422    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/935e18f3-2fcc-4d93-91fa-da3b7f510aa9-tls-certs\") pod \"model-serving-api-54d89bd5fc-cdhx8\" (UID: \"935e18f3-2fcc-4d93-91fa-da3b7f510aa9\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:16.202557 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.202530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ddh2\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-kube-api-access-8ddh2\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.202690 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.202589    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.202828 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.202809    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.202928 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.202852    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.202928 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.202900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.203050 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.202946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.203050 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.203002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.203515 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.203487    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.205490 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.205465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.205651 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.205630    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.206015 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.205988    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.206137 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.206107    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.211622 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.211601    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ddh2\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-kube-api-access-8ddh2\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.212411 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.212393    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hqtxz\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.308596 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.308567    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:16.349436 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.349390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz" event={"ID":"d6ec55d0-2bd3-4922-8f5e-0576651571e1","Type":"ContainerStarted","Data":"b0499c0a248a26a5fdabfdcaca7a74072a1cdae7a20532ad3d72a1cc4691570a"}
Jul 24 17:46:16.363854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.363818    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:16.456116 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.456085    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"]
Jul 24 17:46:16.460172 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:46:16.460141    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14aa01c2_31ff_4f7d_97ff_57b3a6505d94.slice/crio-495cda8bf41925df00f3c574e3877bda621be4da995946b3e39a6c919f9bc3ee WatchSource:0}: Error finding container 495cda8bf41925df00f3c574e3877bda621be4da995946b3e39a6c919f9bc3ee: Status 404 returned error can't find the container with id 495cda8bf41925df00f3c574e3877bda621be4da995946b3e39a6c919f9bc3ee
Jul 24 17:46:16.524566 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:16.524537    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cdhx8"]
Jul 24 17:46:16.526054 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:46:16.526023    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod935e18f3_2fcc_4d93_91fa_da3b7f510aa9.slice/crio-c7b6e7f672009501e6d7cdfce6e3eacc9bd4e1a946706140aa4b441ed63e198e WatchSource:0}: Error finding container c7b6e7f672009501e6d7cdfce6e3eacc9bd4e1a946706140aa4b441ed63e198e: Status 404 returned error can't find the container with id c7b6e7f672009501e6d7cdfce6e3eacc9bd4e1a946706140aa4b441ed63e198e
Jul 24 17:46:17.366034 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:17.365906    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz" event={"ID":"14aa01c2-31ff-4f7d-97ff-57b3a6505d94","Type":"ContainerStarted","Data":"495cda8bf41925df00f3c574e3877bda621be4da995946b3e39a6c919f9bc3ee"}
Jul 24 17:46:17.367708 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:17.367618    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8" event={"ID":"935e18f3-2fcc-4d93-91fa-da3b7f510aa9","Type":"ContainerStarted","Data":"c7b6e7f672009501e6d7cdfce6e3eacc9bd4e1a946706140aa4b441ed63e198e"}
Jul 24 17:46:20.836290 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:20.836242    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:46:20.836652 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:20.836333    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:46:21.396157 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.396118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz" event={"ID":"14aa01c2-31ff-4f7d-97ff-57b3a6505d94","Type":"ContainerStarted","Data":"54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932"}
Jul 24 17:46:21.396358 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.396306    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:21.397821 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.397769    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz" event={"ID":"d58dc331-b5f6-4522-82fd-c56eaa0f150e","Type":"ContainerStarted","Data":"b38c51bc9d0aeeef2069a4b8810aae416d1d03b923748263fb997f240e02d15b"}
Jul 24 17:46:21.398189 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.398163    2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-hqtxz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 17:46:21.398297 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.398217    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz" podUID="14aa01c2-31ff-4f7d-97ff-57b3a6505d94" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 17:46:21.399505 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.399479    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8" event={"ID":"935e18f3-2fcc-4d93-91fa-da3b7f510aa9","Type":"ContainerStarted","Data":"015b6dcf4c94e8ae2b879e851de452ef5fb3d126cebf4d13688cb84c191ec569"}
Jul 24 17:46:21.399618 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.399594    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:46:21.407504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.407480    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz" event={"ID":"d6ec55d0-2bd3-4922-8f5e-0576651571e1","Type":"ContainerStarted","Data":"0cfd0102d874c80e14210b66283c11d589589d4afe44b78f324f753ee98db8c0"}
Jul 24 17:46:21.407586 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.407525    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:21.439605 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.439562    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8" podStartSLOduration=2.19997964 podStartE2EDuration="6.439547899s" totalImagesPullingTime="4.239568259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:21.43853865 +0000 UTC m=+388.643230459" watchObservedRunningTime="2026-07-24 17:46:21.439547899 +0000 UTC m=+388.644239698"
Jul 24 17:46:21.441054 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.441020    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz" podStartSLOduration=2.069499422 podStartE2EDuration="6.441008122s" totalImagesPullingTime="4.3715087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:21.4181388 +0000 UTC m=+388.622830597" watchObservedRunningTime="2026-07-24 17:46:21.441008122 +0000 UTC m=+388.645699919"
Jul 24 17:46:21.457759 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.457713    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-69ddb4868f-2bngz" podStartSLOduration=2.699382932 podStartE2EDuration="7.457696405s" totalImagesPullingTime="4.758313473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:21.455590119 +0000 UTC m=+388.660281917" watchObservedRunningTime="2026-07-24 17:46:21.457696405 +0000 UTC m=+388.662388204"
Jul 24 17:46:21.476705 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:21.476661    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz" podStartSLOduration=2.552899253 podStartE2EDuration="7.476647473s" totalImagesPullingTime="4.92374822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:21.475558347 +0000 UTC m=+388.680250144" watchObservedRunningTime="2026-07-24 17:46:21.476647473 +0000 UTC m=+388.681339269"
Jul 24 17:46:22.406401 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:22.406370    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:25.561600 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.561569    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"]
Jul 24 17:46:25.565236 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.565217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:46:25.567838 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.567819    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 24 17:46:25.568822 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.568808    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-7mf78\""
Jul 24 17:46:25.574807 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.574789    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"]
Jul 24 17:46:25.577988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.577949    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:25.579673 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.579654    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"]
Jul 24 17:46:25.580491 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.580469    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wjphw\""
Jul 24 17:46:25.580987 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.580966    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 24 17:46:25.589491 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.589472    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"]
Jul 24 17:46:25.683257 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.683226    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb9c57db-3607-4779-8af3-91fde3a74002-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fx66p\" (UID: \"fb9c57db-3607-4779-8af3-91fde3a74002\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:25.683389 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.683265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c087d267-61f1-4796-88d5-1564dfe417fc-cert\") pod \"kserve-controller-manager-d6f4b65b8-2jnjw\" (UID: \"c087d267-61f1-4796-88d5-1564dfe417fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:46:25.683389 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.683297    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmz49\" (UniqueName: \"kubernetes.io/projected/fb9c57db-3607-4779-8af3-91fde3a74002-kube-api-access-kmz49\") pod \"llmisvc-controller-manager-777c6bd7d5-fx66p\" (UID: \"fb9c57db-3607-4779-8af3-91fde3a74002\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:25.683389 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.683322    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shdsc\" (UniqueName: \"kubernetes.io/projected/c087d267-61f1-4796-88d5-1564dfe417fc-kube-api-access-shdsc\") pod \"kserve-controller-manager-d6f4b65b8-2jnjw\" (UID: \"c087d267-61f1-4796-88d5-1564dfe417fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:46:25.784360 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.784328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb9c57db-3607-4779-8af3-91fde3a74002-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fx66p\" (UID: \"fb9c57db-3607-4779-8af3-91fde3a74002\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:25.784502 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.784365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c087d267-61f1-4796-88d5-1564dfe417fc-cert\") pod \"kserve-controller-manager-d6f4b65b8-2jnjw\" (UID: \"c087d267-61f1-4796-88d5-1564dfe417fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:46:25.784502 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.784394    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmz49\" (UniqueName: \"kubernetes.io/projected/fb9c57db-3607-4779-8af3-91fde3a74002-kube-api-access-kmz49\") pod \"llmisvc-controller-manager-777c6bd7d5-fx66p\" (UID: \"fb9c57db-3607-4779-8af3-91fde3a74002\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:25.784502 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.784440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shdsc\" (UniqueName: \"kubernetes.io/projected/c087d267-61f1-4796-88d5-1564dfe417fc-kube-api-access-shdsc\") pod \"kserve-controller-manager-d6f4b65b8-2jnjw\" (UID: \"c087d267-61f1-4796-88d5-1564dfe417fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:46:25.784502 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:46:25.784482    2562 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 24 17:46:25.784623 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:46:25.784551    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb9c57db-3607-4779-8af3-91fde3a74002-cert podName:fb9c57db-3607-4779-8af3-91fde3a74002 nodeName:}" failed. No retries permitted until 2026-07-24 17:46:26.284529444 +0000 UTC m=+393.489221241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fb9c57db-3607-4779-8af3-91fde3a74002-cert") pod "llmisvc-controller-manager-777c6bd7d5-fx66p" (UID: "fb9c57db-3607-4779-8af3-91fde3a74002") : secret "llmisvc-webhook-server-cert" not found
Jul 24 17:46:25.786910 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.786857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c087d267-61f1-4796-88d5-1564dfe417fc-cert\") pod \"kserve-controller-manager-d6f4b65b8-2jnjw\" (UID: \"c087d267-61f1-4796-88d5-1564dfe417fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:46:25.802571 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.802544    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shdsc\" (UniqueName: \"kubernetes.io/projected/c087d267-61f1-4796-88d5-1564dfe417fc-kube-api-access-shdsc\") pod \"kserve-controller-manager-d6f4b65b8-2jnjw\" (UID: \"c087d267-61f1-4796-88d5-1564dfe417fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:46:25.807549 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.807520    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmz49\" (UniqueName: \"kubernetes.io/projected/fb9c57db-3607-4779-8af3-91fde3a74002-kube-api-access-kmz49\") pod \"llmisvc-controller-manager-777c6bd7d5-fx66p\" (UID: \"fb9c57db-3607-4779-8af3-91fde3a74002\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:25.875393 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:25.875325    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:46:26.004680 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:26.004655    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"]
Jul 24 17:46:26.005997 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:46:26.005973    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc087d267_61f1_4796_88d5_1564dfe417fc.slice/crio-9961e512cc5293b00e51d3d91e51b0920b3ec0318a7d2c899ff9c0887393a9aa WatchSource:0}: Error finding container 9961e512cc5293b00e51d3d91e51b0920b3ec0318a7d2c899ff9c0887393a9aa: Status 404 returned error can't find the container with id 9961e512cc5293b00e51d3d91e51b0920b3ec0318a7d2c899ff9c0887393a9aa
Jul 24 17:46:26.289475 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:26.289448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb9c57db-3607-4779-8af3-91fde3a74002-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fx66p\" (UID: \"fb9c57db-3607-4779-8af3-91fde3a74002\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:26.291922 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:26.291897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb9c57db-3607-4779-8af3-91fde3a74002-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fx66p\" (UID: \"fb9c57db-3607-4779-8af3-91fde3a74002\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:26.427205 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:26.427172    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw" event={"ID":"c087d267-61f1-4796-88d5-1564dfe417fc","Type":"ContainerStarted","Data":"9961e512cc5293b00e51d3d91e51b0920b3ec0318a7d2c899ff9c0887393a9aa"}
Jul 24 17:46:26.490000 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:26.489975    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:26.636729 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:26.636703    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"]
Jul 24 17:46:26.639156 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:46:26.639127    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfb9c57db_3607_4779_8af3_91fde3a74002.slice/crio-8c876c34505e803a44aef09ec87a0de3fc4fd6f5eb88baad9636e28a8a0c5677 WatchSource:0}: Error finding container 8c876c34505e803a44aef09ec87a0de3fc4fd6f5eb88baad9636e28a8a0c5677: Status 404 returned error can't find the container with id 8c876c34505e803a44aef09ec87a0de3fc4fd6f5eb88baad9636e28a8a0c5677
Jul 24 17:46:27.438769 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:27.438738    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p" event={"ID":"fb9c57db-3607-4779-8af3-91fde3a74002","Type":"ContainerStarted","Data":"8c876c34505e803a44aef09ec87a0de3fc4fd6f5eb88baad9636e28a8a0c5677"}
Jul 24 17:46:29.454711 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:29.454008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw" event={"ID":"c087d267-61f1-4796-88d5-1564dfe417fc","Type":"ContainerStarted","Data":"056f5d01170719dd75077b58792fdc9ba259e537854e89d8ad7d2f61dbcc57c5"}
Jul 24 17:46:29.454711 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:29.454661    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:46:29.456945 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:29.456745    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"]
Jul 24 17:46:29.457073 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:29.457015    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz" podUID="14aa01c2-31ff-4f7d-97ff-57b3a6505d94" containerName="discovery" containerID="cri-o://54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932" gracePeriod=30
Jul 24 17:46:29.471500 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:29.471448    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw" podStartSLOduration=2.029649067 podStartE2EDuration="4.471428844s" totalImagesPullingTime="2.441779777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:29.470620469 +0000 UTC m=+396.675312269" watchObservedRunningTime="2026-07-24 17:46:29.471428844 +0000 UTC m=+396.676120620"
Jul 24 17:46:30.114001 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.113979    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:30.224374 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.224310    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-dns-cert\") pod \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") "
Jul 24 17:46:30.224374 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.224365    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-ca-configmap\") pod \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") "
Jul 24 17:46:30.224525 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.224402    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-cacerts\") pod \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") "
Jul 24 17:46:30.224525 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.224431    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-kubeconfig\") pod \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") "
Jul 24 17:46:30.224525 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.224455    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-token\") pod \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") "
Jul 24 17:46:30.224525 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.224477    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ddh2\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-kube-api-access-8ddh2\") pod \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") "
Jul 24 17:46:30.224525 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.224508    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-local-certs\") pod \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\" (UID: \"14aa01c2-31ff-4f7d-97ff-57b3a6505d94\") "
Jul 24 17:46:30.224862 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.224828    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "14aa01c2-31ff-4f7d-97ff-57b3a6505d94" (UID: "14aa01c2-31ff-4f7d-97ff-57b3a6505d94"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:46:30.226747 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.226713    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "14aa01c2-31ff-4f7d-97ff-57b3a6505d94" (UID: "14aa01c2-31ff-4f7d-97ff-57b3a6505d94"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:46:30.226850 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.226786    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-cacerts" (OuterVolumeSpecName: "cacerts") pod "14aa01c2-31ff-4f7d-97ff-57b3a6505d94" (UID: "14aa01c2-31ff-4f7d-97ff-57b3a6505d94"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:46:30.226934 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.226839    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "14aa01c2-31ff-4f7d-97ff-57b3a6505d94" (UID: "14aa01c2-31ff-4f7d-97ff-57b3a6505d94"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:46:30.226993 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.226965    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-token" (OuterVolumeSpecName: "istio-token") pod "14aa01c2-31ff-4f7d-97ff-57b3a6505d94" (UID: "14aa01c2-31ff-4f7d-97ff-57b3a6505d94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:46:30.227050 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.227007    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-kube-api-access-8ddh2" (OuterVolumeSpecName: "kube-api-access-8ddh2") pod "14aa01c2-31ff-4f7d-97ff-57b3a6505d94" (UID: "14aa01c2-31ff-4f7d-97ff-57b3a6505d94"). InnerVolumeSpecName "kube-api-access-8ddh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:46:30.227103 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.227075    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-local-certs" (OuterVolumeSpecName: "local-certs") pod "14aa01c2-31ff-4f7d-97ff-57b3a6505d94" (UID: "14aa01c2-31ff-4f7d-97ff-57b3a6505d94"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:46:30.325609 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.325578    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-token\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:46:30.325609 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.325605    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ddh2\" (UniqueName: \"kubernetes.io/projected/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-kube-api-access-8ddh2\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:46:30.325609 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.325615    2562 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-local-certs\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:46:30.325792 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.325625    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-dns-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:46:30.325792 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.325638    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-csr-ca-configmap\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:46:30.325792 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.325650    2562 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-cacerts\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:46:30.325792 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.325658    2562 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14aa01c2-31ff-4f7d-97ff-57b3a6505d94-istio-kubeconfig\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:46:30.460122 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.460083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p" event={"ID":"fb9c57db-3607-4779-8af3-91fde3a74002","Type":"ContainerStarted","Data":"1036703c40dc13146b9745951ec5ae5c60321c883e8c05c04f8d90b2b301d7d0"}
Jul 24 17:46:30.460539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.460154    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:46:30.461521 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.461497    2562 generic.go:356] "Generic (PLEG): container finished" podID="14aa01c2-31ff-4f7d-97ff-57b3a6505d94" containerID="54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932" exitCode=0
Jul 24 17:46:30.461628 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.461534    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"
Jul 24 17:46:30.461628 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.461577    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz" event={"ID":"14aa01c2-31ff-4f7d-97ff-57b3a6505d94","Type":"ContainerDied","Data":"54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932"}
Jul 24 17:46:30.461628 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.461620    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz" event={"ID":"14aa01c2-31ff-4f7d-97ff-57b3a6505d94","Type":"ContainerDied","Data":"495cda8bf41925df00f3c574e3877bda621be4da995946b3e39a6c919f9bc3ee"}
Jul 24 17:46:30.461762 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.461645    2562 scope.go:117] "RemoveContainer" containerID="54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932"
Jul 24 17:46:30.472153 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.472133    2562 scope.go:117] "RemoveContainer" containerID="54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932"
Jul 24 17:46:30.472426 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:46:30.472406    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932\": container with ID starting with 54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932 not found: ID does not exist" containerID="54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932"
Jul 24 17:46:30.472486 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.472433    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932"} err="failed to get container status \"54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932\": rpc error: code = NotFound desc = could not find container \"54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932\": container with ID starting with 54ee8776b0416dd50b25cb27c257343850a68d8499a60427a4ad9471f104e932 not found: ID does not exist"
Jul 24 17:46:30.480967 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.480891    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p" podStartSLOduration=2.099732901 podStartE2EDuration="5.480863649s" totalImagesPullingTime="3.381130748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:30.479382543 +0000 UTC m=+397.684074340" watchObservedRunningTime="2026-07-24 17:46:30.480863649 +0000 UTC m=+397.685555445"
Jul 24 17:46:30.494526 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.494481    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"]
Jul 24 17:46:30.496362 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:30.496342    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hqtxz"]
Jul 24 17:46:31.407739 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:31.407705    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14aa01c2-31ff-4f7d-97ff-57b3a6505d94" path="/var/lib/kubelet/pods/14aa01c2-31ff-4f7d-97ff-57b3a6505d94/volumes"
Jul 24 17:46:32.408568 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:32.408533    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gqrlz"
Jul 24 17:46:32.410328 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:46:32.410306    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cdhx8"
Jul 24 17:47:00.467250 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:00.467218    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2jnjw"
Jul 24 17:47:01.468367 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:01.468336    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fx66p"
Jul 24 17:47:15.734145 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.734063    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2mrx"]
Jul 24 17:47:15.734609 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.734544    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14aa01c2-31ff-4f7d-97ff-57b3a6505d94" containerName="discovery"
Jul 24 17:47:15.734609 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.734571    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="14aa01c2-31ff-4f7d-97ff-57b3a6505d94" containerName="discovery"
Jul 24 17:47:15.734736 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.734686    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="14aa01c2-31ff-4f7d-97ff-57b3a6505d94" containerName="discovery"
Jul 24 17:47:15.750566 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.750535    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2mrx"]
Jul 24 17:47:15.750704 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.750638    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx"
Jul 24 17:47:15.754041 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.754017    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cltwp\""
Jul 24 17:47:15.754147 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.754060    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 17:47:15.755300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.755114    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 17:47:15.771501 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.771479    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4v2z\" (UniqueName: \"kubernetes.io/projected/d63419ec-edf0-40d0-a385-a05ab6bb2d76-kube-api-access-j4v2z\") pod \"kuadrant-operator-catalog-b2mrx\" (UID: \"d63419ec-edf0-40d0-a385-a05ab6bb2d76\") " pod="kuadrant-system/kuadrant-operator-catalog-b2mrx"
Jul 24 17:47:15.871965 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.871942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4v2z\" (UniqueName: \"kubernetes.io/projected/d63419ec-edf0-40d0-a385-a05ab6bb2d76-kube-api-access-j4v2z\") pod \"kuadrant-operator-catalog-b2mrx\" (UID: \"d63419ec-edf0-40d0-a385-a05ab6bb2d76\") " pod="kuadrant-system/kuadrant-operator-catalog-b2mrx"
Jul 24 17:47:15.880354 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:15.880333    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4v2z\" (UniqueName: \"kubernetes.io/projected/d63419ec-edf0-40d0-a385-a05ab6bb2d76-kube-api-access-j4v2z\") pod \"kuadrant-operator-catalog-b2mrx\" (UID: \"d63419ec-edf0-40d0-a385-a05ab6bb2d76\") " pod="kuadrant-system/kuadrant-operator-catalog-b2mrx"
Jul 24 17:47:16.062056 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.062031    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx"
Jul 24 17:47:16.094437 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.094404    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2mrx"]
Jul 24 17:47:16.190850 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.190825    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2mrx"]
Jul 24 17:47:16.193399 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:16.193373    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd63419ec_edf0_40d0_a385_a05ab6bb2d76.slice/crio-8c46fe0396ff40b6ab3063e3da120cf38ac46302c8db5f9fb20a975c1d888084 WatchSource:0}: Error finding container 8c46fe0396ff40b6ab3063e3da120cf38ac46302c8db5f9fb20a975c1d888084: Status 404 returned error can't find the container with id 8c46fe0396ff40b6ab3063e3da120cf38ac46302c8db5f9fb20a975c1d888084
Jul 24 17:47:16.316467 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.316414    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-56vqg"]
Jul 24 17:47:16.320669 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.320652    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-56vqg"
Jul 24 17:47:16.343471 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.343449    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-56vqg"]
Jul 24 17:47:16.376074 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.376049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m25ct\" (UniqueName: \"kubernetes.io/projected/91411414-1e88-46dc-bf85-1c1f5d6d146a-kube-api-access-m25ct\") pod \"kuadrant-operator-catalog-56vqg\" (UID: \"91411414-1e88-46dc-bf85-1c1f5d6d146a\") " pod="kuadrant-system/kuadrant-operator-catalog-56vqg"
Jul 24 17:47:16.476417 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.476395    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m25ct\" (UniqueName: \"kubernetes.io/projected/91411414-1e88-46dc-bf85-1c1f5d6d146a-kube-api-access-m25ct\") pod \"kuadrant-operator-catalog-56vqg\" (UID: \"91411414-1e88-46dc-bf85-1c1f5d6d146a\") " pod="kuadrant-system/kuadrant-operator-catalog-56vqg"
Jul 24 17:47:16.485337 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.485315    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m25ct\" (UniqueName: \"kubernetes.io/projected/91411414-1e88-46dc-bf85-1c1f5d6d146a-kube-api-access-m25ct\") pod \"kuadrant-operator-catalog-56vqg\" (UID: \"91411414-1e88-46dc-bf85-1c1f5d6d146a\") " pod="kuadrant-system/kuadrant-operator-catalog-56vqg"
Jul 24 17:47:16.630101 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.630027    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-56vqg"
Jul 24 17:47:16.683495 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.683463    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx" event={"ID":"d63419ec-edf0-40d0-a385-a05ab6bb2d76","Type":"ContainerStarted","Data":"8c46fe0396ff40b6ab3063e3da120cf38ac46302c8db5f9fb20a975c1d888084"}
Jul 24 17:47:16.782396 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:16.782369    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-56vqg"]
Jul 24 17:47:16.784064 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:16.784037    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91411414_1e88_46dc_bf85_1c1f5d6d146a.slice/crio-726caaea51d3ab29eed8d3a5c2e4fbe2917a924af95f294b3e1245fa5a95f4ae WatchSource:0}: Error finding container 726caaea51d3ab29eed8d3a5c2e4fbe2917a924af95f294b3e1245fa5a95f4ae: Status 404 returned error can't find the container with id 726caaea51d3ab29eed8d3a5c2e4fbe2917a924af95f294b3e1245fa5a95f4ae
Jul 24 17:47:17.692754 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:17.692723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-56vqg" event={"ID":"91411414-1e88-46dc-bf85-1c1f5d6d146a","Type":"ContainerStarted","Data":"726caaea51d3ab29eed8d3a5c2e4fbe2917a924af95f294b3e1245fa5a95f4ae"}
Jul 24 17:47:18.698631 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:18.698534    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx" event={"ID":"d63419ec-edf0-40d0-a385-a05ab6bb2d76","Type":"ContainerStarted","Data":"e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205"}
Jul 24 17:47:18.698631 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:18.698562    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx" podUID="d63419ec-edf0-40d0-a385-a05ab6bb2d76" containerName="registry-server" containerID="cri-o://e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205" gracePeriod=2
Jul 24 17:47:18.700173 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:18.700149    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-56vqg" event={"ID":"91411414-1e88-46dc-bf85-1c1f5d6d146a","Type":"ContainerStarted","Data":"e0878e4506e73952167d47b0056a1f1a388f1dea349293a07624f8ce8b427bf5"}
Jul 24 17:47:18.715689 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:18.715651    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx" podStartSLOduration=1.568132327 podStartE2EDuration="3.715639069s" totalImagesPullingTime="2.147506742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:18.714872616 +0000 UTC m=+445.919564407" watchObservedRunningTime="2026-07-24 17:47:18.715639069 +0000 UTC m=+445.920330865"
Jul 24 17:47:18.736282 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:18.734258    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-56vqg" podStartSLOduration=1.176510772 podStartE2EDuration="2.734241143s" totalImagesPullingTime="1.557730371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:18.730477172 +0000 UTC m=+445.935168971" watchObservedRunningTime="2026-07-24 17:47:18.734241143 +0000 UTC m=+445.938932945"
Jul 24 17:47:18.956744 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:18.956689    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx"
Jul 24 17:47:18.995729 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:18.995703    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4v2z\" (UniqueName: \"kubernetes.io/projected/d63419ec-edf0-40d0-a385-a05ab6bb2d76-kube-api-access-j4v2z\") pod \"d63419ec-edf0-40d0-a385-a05ab6bb2d76\" (UID: \"d63419ec-edf0-40d0-a385-a05ab6bb2d76\") "
Jul 24 17:47:18.997752 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:18.997723    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d63419ec-edf0-40d0-a385-a05ab6bb2d76-kube-api-access-j4v2z" (OuterVolumeSpecName: "kube-api-access-j4v2z") pod "d63419ec-edf0-40d0-a385-a05ab6bb2d76" (UID: "d63419ec-edf0-40d0-a385-a05ab6bb2d76"). InnerVolumeSpecName "kube-api-access-j4v2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:19.096422 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.096395    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4v2z\" (UniqueName: \"kubernetes.io/projected/d63419ec-edf0-40d0-a385-a05ab6bb2d76-kube-api-access-j4v2z\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:19.706320 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.706234    2562 generic.go:356] "Generic (PLEG): container finished" podID="d63419ec-edf0-40d0-a385-a05ab6bb2d76" containerID="e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205" exitCode=0
Jul 24 17:47:19.706320 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.706273    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx"
Jul 24 17:47:19.706801 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.706326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx" event={"ID":"d63419ec-edf0-40d0-a385-a05ab6bb2d76","Type":"ContainerDied","Data":"e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205"}
Jul 24 17:47:19.706801 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.706388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2mrx" event={"ID":"d63419ec-edf0-40d0-a385-a05ab6bb2d76","Type":"ContainerDied","Data":"8c46fe0396ff40b6ab3063e3da120cf38ac46302c8db5f9fb20a975c1d888084"}
Jul 24 17:47:19.706801 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.706418    2562 scope.go:117] "RemoveContainer" containerID="e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205"
Jul 24 17:47:19.715752 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.715735    2562 scope.go:117] "RemoveContainer" containerID="e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205"
Jul 24 17:47:19.715992 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:47:19.715974    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205\": container with ID starting with e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205 not found: ID does not exist" containerID="e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205"
Jul 24 17:47:19.716037 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.716000    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205"} err="failed to get container status \"e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205\": rpc error: code = NotFound desc = could not find container \"e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205\": container with ID starting with e7bf360fb500f5ed52bddf5a9626adc81517c2a0bcbcc6c6321e1b807540c205 not found: ID does not exist"
Jul 24 17:47:19.724135 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.724118    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2mrx"]
Jul 24 17:47:19.725838 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:19.725818    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2mrx"]
Jul 24 17:47:21.407635 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:21.407604    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d63419ec-edf0-40d0-a385-a05ab6bb2d76" path="/var/lib/kubelet/pods/d63419ec-edf0-40d0-a385-a05ab6bb2d76/volumes"
Jul 24 17:47:26.630318 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:26.630272    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-56vqg"
Jul 24 17:47:26.630773 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:26.630338    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-56vqg"
Jul 24 17:47:26.653114 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:26.653091    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-56vqg"
Jul 24 17:47:26.761467 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:26.761444    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-56vqg"
Jul 24 17:47:28.341671 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.341632    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"]
Jul 24 17:47:28.342174 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.342131    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d63419ec-edf0-40d0-a385-a05ab6bb2d76" containerName="registry-server"
Jul 24 17:47:28.342174 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.342149    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d63419ec-edf0-40d0-a385-a05ab6bb2d76" containerName="registry-server"
Jul 24 17:47:28.342287 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.342267    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d63419ec-edf0-40d0-a385-a05ab6bb2d76" containerName="registry-server"
Jul 24 17:47:28.346797 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.346774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.349320 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.349301    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gd6n5\""
Jul 24 17:47:28.352765 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.352740    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"]
Jul 24 17:47:28.366572 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.366540    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.366694 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.366577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.366694 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.366622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vbw8\" (UniqueName: \"kubernetes.io/projected/9bc9c580-1d34-4ee0-b364-173f8bc862b7-kube-api-access-9vbw8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.467092 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.467068    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vbw8\" (UniqueName: \"kubernetes.io/projected/9bc9c580-1d34-4ee0-b364-173f8bc862b7-kube-api-access-9vbw8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.467187 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.467134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.467270 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.467245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.467525 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.467505    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.467565 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.467509    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.476133 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.476106    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vbw8\" (UniqueName: \"kubernetes.io/projected/9bc9c580-1d34-4ee0-b364-173f8bc862b7-kube-api-access-9vbw8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.657621 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.657562    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:28.786854 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.786829    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"]
Jul 24 17:47:28.788805 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:28.788776    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc9c580_1d34_4ee0_b364_173f8bc862b7.slice/crio-6e811da42ef00f174efde22e64e5ddcdd1bf7dd25ccd0b373f80484a7afb7b5d WatchSource:0}: Error finding container 6e811da42ef00f174efde22e64e5ddcdd1bf7dd25ccd0b373f80484a7afb7b5d: Status 404 returned error can't find the container with id 6e811da42ef00f174efde22e64e5ddcdd1bf7dd25ccd0b373f80484a7afb7b5d
Jul 24 17:47:28.939249 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.939197    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"]
Jul 24 17:47:28.942742 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.942725    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:28.950541 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.950521    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"]
Jul 24 17:47:28.971600 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.971570    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:28.971685 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.971667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:28.971725 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:28.971707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jcrb\" (UniqueName: \"kubernetes.io/projected/897af391-380d-4470-9f9b-297c14483c2a-kube-api-access-6jcrb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:29.073040 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.073016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jcrb\" (UniqueName: \"kubernetes.io/projected/897af391-380d-4470-9f9b-297c14483c2a-kube-api-access-6jcrb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:29.073131 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.073049    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:29.073131 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.073109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:29.073484 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.073464    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:29.073524 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.073482    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:29.081092 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.081067    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jcrb\" (UniqueName: \"kubernetes.io/projected/897af391-380d-4470-9f9b-297c14483c2a-kube-api-access-6jcrb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:29.251743 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.251680    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:29.415496 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.415474    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"]
Jul 24 17:47:29.417067 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:29.417041    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897af391_380d_4470_9f9b_297c14483c2a.slice/crio-59f24676493b7018ebdeca78565011ec027dc47cb3ff6e87788e86ce838b6acd WatchSource:0}: Error finding container 59f24676493b7018ebdeca78565011ec027dc47cb3ff6e87788e86ce838b6acd: Status 404 returned error can't find the container with id 59f24676493b7018ebdeca78565011ec027dc47cb3ff6e87788e86ce838b6acd
Jul 24 17:47:29.561779 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.561716    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"]
Jul 24 17:47:29.567259 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.567239    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.576260 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.576208    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"]
Jul 24 17:47:29.677797 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.677773    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.677917 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.677805    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjp8c\" (UniqueName: \"kubernetes.io/projected/311055f3-f587-4d42-a8bc-e549b2817d89-kube-api-access-mjp8c\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.677917 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.677829    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.757359 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.757306    2562 generic.go:356] "Generic (PLEG): container finished" podID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerID="ebac2818a31cc3ef7143a532465d39aaf41d749a67b966e9902116ed495a5cd5" exitCode=0
Jul 24 17:47:29.757452 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.757388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84" event={"ID":"9bc9c580-1d34-4ee0-b364-173f8bc862b7","Type":"ContainerDied","Data":"ebac2818a31cc3ef7143a532465d39aaf41d749a67b966e9902116ed495a5cd5"}
Jul 24 17:47:29.757452 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.757430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84" event={"ID":"9bc9c580-1d34-4ee0-b364-173f8bc862b7","Type":"ContainerStarted","Data":"6e811da42ef00f174efde22e64e5ddcdd1bf7dd25ccd0b373f80484a7afb7b5d"}
Jul 24 17:47:29.759030 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.759008    2562 generic.go:356] "Generic (PLEG): container finished" podID="897af391-380d-4470-9f9b-297c14483c2a" containerID="c737a4645f4a2b494e30f7dfd5c33ee4c930d5fc012417c742fa483a8087b211" exitCode=0
Jul 24 17:47:29.759149 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.759041    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9" event={"ID":"897af391-380d-4470-9f9b-297c14483c2a","Type":"ContainerDied","Data":"c737a4645f4a2b494e30f7dfd5c33ee4c930d5fc012417c742fa483a8087b211"}
Jul 24 17:47:29.759149 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.759106    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9" event={"ID":"897af391-380d-4470-9f9b-297c14483c2a","Type":"ContainerStarted","Data":"59f24676493b7018ebdeca78565011ec027dc47cb3ff6e87788e86ce838b6acd"}
Jul 24 17:47:29.778683 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.778662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.778773 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.778690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjp8c\" (UniqueName: \"kubernetes.io/projected/311055f3-f587-4d42-a8bc-e549b2817d89-kube-api-access-mjp8c\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.778773 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.778725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.779035 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.779017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.779120 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.779083    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.787096 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.787077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjp8c\" (UniqueName: \"kubernetes.io/projected/311055f3-f587-4d42-a8bc-e549b2817d89-kube-api-access-mjp8c\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:29.877441 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:29.877411    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:30.008081 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.008056    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"]
Jul 24 17:47:30.009661 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:30.009636    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311055f3_f587_4d42_a8bc_e549b2817d89.slice/crio-a45dcf3b0ecf937761cebcf7f200716fcd07cd64d57dacef3b2fdba8e3f63900 WatchSource:0}: Error finding container a45dcf3b0ecf937761cebcf7f200716fcd07cd64d57dacef3b2fdba8e3f63900: Status 404 returned error can't find the container with id a45dcf3b0ecf937761cebcf7f200716fcd07cd64d57dacef3b2fdba8e3f63900
Jul 24 17:47:30.141321 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.141297    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"]
Jul 24 17:47:30.144936 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.144918    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.152828 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.152808    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"]
Jul 24 17:47:30.182752 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.182731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.182837 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.182809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jdjl\" (UniqueName: \"kubernetes.io/projected/66aa18ba-854e-498e-8469-4721f4664c8e-kube-api-access-6jdjl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.182908 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.182859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.284218 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.284165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jdjl\" (UniqueName: \"kubernetes.io/projected/66aa18ba-854e-498e-8469-4721f4664c8e-kube-api-access-6jdjl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.284218 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.284211    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.284356 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.284244    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.284608 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.284589    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.284651 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.284605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.293004 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.292982    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jdjl\" (UniqueName: \"kubernetes.io/projected/66aa18ba-854e-498e-8469-4721f4664c8e-kube-api-access-6jdjl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.465159 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.465135    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:30.607604 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.607580    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"]
Jul 24 17:47:30.609156 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:30.609131    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66aa18ba_854e_498e_8469_4721f4664c8e.slice/crio-9403dd781285efdb71a4ceef9d21bf0c3999ef8815fa8efafa68666f0289ec29 WatchSource:0}: Error finding container 9403dd781285efdb71a4ceef9d21bf0c3999ef8815fa8efafa68666f0289ec29: Status 404 returned error can't find the container with id 9403dd781285efdb71a4ceef9d21bf0c3999ef8815fa8efafa68666f0289ec29
Jul 24 17:47:30.766029 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.765991    2562 generic.go:356] "Generic (PLEG): container finished" podID="897af391-380d-4470-9f9b-297c14483c2a" containerID="01e2d67952bb775e406b23fb08cf11b6833479dccccab08b3bc00f7b3928f766" exitCode=0
Jul 24 17:47:30.766150 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.766056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9" event={"ID":"897af391-380d-4470-9f9b-297c14483c2a","Type":"ContainerDied","Data":"01e2d67952bb775e406b23fb08cf11b6833479dccccab08b3bc00f7b3928f766"}
Jul 24 17:47:30.767958 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.767934    2562 generic.go:356] "Generic (PLEG): container finished" podID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerID="7f3eddbe331a5268ab51a3e4f9341e2010efcf845995b9a59e5a7f7411c2719e" exitCode=0
Jul 24 17:47:30.768049 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.768026    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84" event={"ID":"9bc9c580-1d34-4ee0-b364-173f8bc862b7","Type":"ContainerDied","Data":"7f3eddbe331a5268ab51a3e4f9341e2010efcf845995b9a59e5a7f7411c2719e"}
Jul 24 17:47:30.769683 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.769662    2562 generic.go:356] "Generic (PLEG): container finished" podID="66aa18ba-854e-498e-8469-4721f4664c8e" containerID="6b6dfec81db97a8e7aad39c0b64ba66324666e14a042fc4569f20367d937cb00" exitCode=0
Jul 24 17:47:30.769774 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.769729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw" event={"ID":"66aa18ba-854e-498e-8469-4721f4664c8e","Type":"ContainerDied","Data":"6b6dfec81db97a8e7aad39c0b64ba66324666e14a042fc4569f20367d937cb00"}
Jul 24 17:47:30.769823 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.769785    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw" event={"ID":"66aa18ba-854e-498e-8469-4721f4664c8e","Type":"ContainerStarted","Data":"9403dd781285efdb71a4ceef9d21bf0c3999ef8815fa8efafa68666f0289ec29"}
Jul 24 17:47:30.771464 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.771425    2562 generic.go:356] "Generic (PLEG): container finished" podID="311055f3-f587-4d42-a8bc-e549b2817d89" containerID="be62acca072495cc83d42a6e3e86fd94dde52856ddf69ddab9e9b91ae350fefb" exitCode=0
Jul 24 17:47:30.771464 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.771450    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr" event={"ID":"311055f3-f587-4d42-a8bc-e549b2817d89","Type":"ContainerDied","Data":"be62acca072495cc83d42a6e3e86fd94dde52856ddf69ddab9e9b91ae350fefb"}
Jul 24 17:47:30.771592 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:30.771489    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr" event={"ID":"311055f3-f587-4d42-a8bc-e549b2817d89","Type":"ContainerStarted","Data":"a45dcf3b0ecf937761cebcf7f200716fcd07cd64d57dacef3b2fdba8e3f63900"}
Jul 24 17:47:31.778952 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:31.778918    2562 generic.go:356] "Generic (PLEG): container finished" podID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerID="a73e5eba06cdb43776c9d59520aeb2c5d6c9a04b36ba43eb29320d01e3c5cdb7" exitCode=0
Jul 24 17:47:31.779248 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:31.778999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84" event={"ID":"9bc9c580-1d34-4ee0-b364-173f8bc862b7","Type":"ContainerDied","Data":"a73e5eba06cdb43776c9d59520aeb2c5d6c9a04b36ba43eb29320d01e3c5cdb7"}
Jul 24 17:47:31.780666 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:31.780644    2562 generic.go:356] "Generic (PLEG): container finished" podID="66aa18ba-854e-498e-8469-4721f4664c8e" containerID="5ce4d5d5aff6dcc4f5707a8ae5153fe5171559af016724390884904ab9bc6de2" exitCode=0
Jul 24 17:47:31.780765 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:31.780713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw" event={"ID":"66aa18ba-854e-498e-8469-4721f4664c8e","Type":"ContainerDied","Data":"5ce4d5d5aff6dcc4f5707a8ae5153fe5171559af016724390884904ab9bc6de2"}
Jul 24 17:47:31.782525 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:31.782502    2562 generic.go:356] "Generic (PLEG): container finished" podID="311055f3-f587-4d42-a8bc-e549b2817d89" containerID="c3c955183327958bc2ce566ae7d99b85b92229745ba15aebdae17430cd028597" exitCode=0
Jul 24 17:47:31.782598 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:31.782579    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr" event={"ID":"311055f3-f587-4d42-a8bc-e549b2817d89","Type":"ContainerDied","Data":"c3c955183327958bc2ce566ae7d99b85b92229745ba15aebdae17430cd028597"}
Jul 24 17:47:31.784950 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:31.784931    2562 generic.go:356] "Generic (PLEG): container finished" podID="897af391-380d-4470-9f9b-297c14483c2a" containerID="2e401e270e1b1fdca8d6bd50bf9ad200ac1b411fba6ead8fa78160a70f182162" exitCode=0
Jul 24 17:47:31.785026 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:31.785006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9" event={"ID":"897af391-380d-4470-9f9b-297c14483c2a","Type":"ContainerDied","Data":"2e401e270e1b1fdca8d6bd50bf9ad200ac1b411fba6ead8fa78160a70f182162"}
Jul 24 17:47:32.793721 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:32.793684    2562 generic.go:356] "Generic (PLEG): container finished" podID="66aa18ba-854e-498e-8469-4721f4664c8e" containerID="8a67f0d19e78340ef96c78dcd35841d299bee5838a2d950530ccc5c43ef310c3" exitCode=0
Jul 24 17:47:32.794216 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:32.793779    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw" event={"ID":"66aa18ba-854e-498e-8469-4721f4664c8e","Type":"ContainerDied","Data":"8a67f0d19e78340ef96c78dcd35841d299bee5838a2d950530ccc5c43ef310c3"}
Jul 24 17:47:32.796115 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:32.796084    2562 generic.go:356] "Generic (PLEG): container finished" podID="311055f3-f587-4d42-a8bc-e549b2817d89" containerID="294fa2d77c836191fc0349bb785699676db7fe7192cd027c6e8b3714e18bc8c3" exitCode=0
Jul 24 17:47:32.796257 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:32.796137    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr" event={"ID":"311055f3-f587-4d42-a8bc-e549b2817d89","Type":"ContainerDied","Data":"294fa2d77c836191fc0349bb785699676db7fe7192cd027c6e8b3714e18bc8c3"}
Jul 24 17:47:32.935064 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:32.935041    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:32.969518 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:32.969493    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:33.008860 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.008835    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-bundle\") pod \"897af391-380d-4470-9f9b-297c14483c2a\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") "
Jul 24 17:47:33.008994 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.008889    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-util\") pod \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") "
Jul 24 17:47:33.008994 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.008907    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jcrb\" (UniqueName: \"kubernetes.io/projected/897af391-380d-4470-9f9b-297c14483c2a-kube-api-access-6jcrb\") pod \"897af391-380d-4470-9f9b-297c14483c2a\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") "
Jul 24 17:47:33.008994 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.008960    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-bundle\") pod \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") "
Jul 24 17:47:33.009158 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.009002    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vbw8\" (UniqueName: \"kubernetes.io/projected/9bc9c580-1d34-4ee0-b364-173f8bc862b7-kube-api-access-9vbw8\") pod \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\" (UID: \"9bc9c580-1d34-4ee0-b364-173f8bc862b7\") "
Jul 24 17:47:33.009158 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.009026    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-util\") pod \"897af391-380d-4470-9f9b-297c14483c2a\" (UID: \"897af391-380d-4470-9f9b-297c14483c2a\") "
Jul 24 17:47:33.009578 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.009555    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-bundle" (OuterVolumeSpecName: "bundle") pod "9bc9c580-1d34-4ee0-b364-173f8bc862b7" (UID: "9bc9c580-1d34-4ee0-b364-173f8bc862b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:33.009968 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.009939    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-bundle" (OuterVolumeSpecName: "bundle") pod "897af391-380d-4470-9f9b-297c14483c2a" (UID: "897af391-380d-4470-9f9b-297c14483c2a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:33.011398 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.011373    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897af391-380d-4470-9f9b-297c14483c2a-kube-api-access-6jcrb" (OuterVolumeSpecName: "kube-api-access-6jcrb") pod "897af391-380d-4470-9f9b-297c14483c2a" (UID: "897af391-380d-4470-9f9b-297c14483c2a"). InnerVolumeSpecName "kube-api-access-6jcrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:33.011653 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.011636    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bc9c580-1d34-4ee0-b364-173f8bc862b7-kube-api-access-9vbw8" (OuterVolumeSpecName: "kube-api-access-9vbw8") pod "9bc9c580-1d34-4ee0-b364-173f8bc862b7" (UID: "9bc9c580-1d34-4ee0-b364-173f8bc862b7"). InnerVolumeSpecName "kube-api-access-9vbw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:33.015622 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.015603    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-util" (OuterVolumeSpecName: "util") pod "9bc9c580-1d34-4ee0-b364-173f8bc862b7" (UID: "9bc9c580-1d34-4ee0-b364-173f8bc862b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:33.017312 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.017288    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-util" (OuterVolumeSpecName: "util") pod "897af391-380d-4470-9f9b-297c14483c2a" (UID: "897af391-380d-4470-9f9b-297c14483c2a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:33.110272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.110224    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-util\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:33.110272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.110245    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jcrb\" (UniqueName: \"kubernetes.io/projected/897af391-380d-4470-9f9b-297c14483c2a-kube-api-access-6jcrb\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:33.110272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.110255    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc9c580-1d34-4ee0-b364-173f8bc862b7-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:33.110272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.110263    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vbw8\" (UniqueName: \"kubernetes.io/projected/9bc9c580-1d34-4ee0-b364-173f8bc862b7-kube-api-access-9vbw8\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:33.110272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.110271    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-util\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:33.110470 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.110280    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/897af391-380d-4470-9f9b-297c14483c2a-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:33.802673 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.802647    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9"
Jul 24 17:47:33.803031 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.802672    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9" event={"ID":"897af391-380d-4470-9f9b-297c14483c2a","Type":"ContainerDied","Data":"59f24676493b7018ebdeca78565011ec027dc47cb3ff6e87788e86ce838b6acd"}
Jul 24 17:47:33.803031 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.802714    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59f24676493b7018ebdeca78565011ec027dc47cb3ff6e87788e86ce838b6acd"
Jul 24 17:47:33.804931 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.804913    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84"
Jul 24 17:47:33.805032 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.804913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84" event={"ID":"9bc9c580-1d34-4ee0-b364-173f8bc862b7","Type":"ContainerDied","Data":"6e811da42ef00f174efde22e64e5ddcdd1bf7dd25ccd0b373f80484a7afb7b5d"}
Jul 24 17:47:33.805032 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.804971    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e811da42ef00f174efde22e64e5ddcdd1bf7dd25ccd0b373f80484a7afb7b5d"
Jul 24 17:47:33.966591 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.966571    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:33.968532 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:33.968517    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:34.018502 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.018478    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jdjl\" (UniqueName: \"kubernetes.io/projected/66aa18ba-854e-498e-8469-4721f4664c8e-kube-api-access-6jdjl\") pod \"66aa18ba-854e-498e-8469-4721f4664c8e\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") "
Jul 24 17:47:34.018629 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.018521    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-util\") pod \"311055f3-f587-4d42-a8bc-e549b2817d89\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") "
Jul 24 17:47:34.018629 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.018564    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-bundle\") pod \"311055f3-f587-4d42-a8bc-e549b2817d89\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") "
Jul 24 17:47:34.018629 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.018600    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjp8c\" (UniqueName: \"kubernetes.io/projected/311055f3-f587-4d42-a8bc-e549b2817d89-kube-api-access-mjp8c\") pod \"311055f3-f587-4d42-a8bc-e549b2817d89\" (UID: \"311055f3-f587-4d42-a8bc-e549b2817d89\") "
Jul 24 17:47:34.018773 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.018672    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-util\") pod \"66aa18ba-854e-498e-8469-4721f4664c8e\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") "
Jul 24 17:47:34.018773 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.018699    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-bundle\") pod \"66aa18ba-854e-498e-8469-4721f4664c8e\" (UID: \"66aa18ba-854e-498e-8469-4721f4664c8e\") "
Jul 24 17:47:34.019222 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.019195    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-bundle" (OuterVolumeSpecName: "bundle") pod "311055f3-f587-4d42-a8bc-e549b2817d89" (UID: "311055f3-f587-4d42-a8bc-e549b2817d89"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:34.019381 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.019355    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-bundle" (OuterVolumeSpecName: "bundle") pod "66aa18ba-854e-498e-8469-4721f4664c8e" (UID: "66aa18ba-854e-498e-8469-4721f4664c8e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:34.020933 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.020910    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66aa18ba-854e-498e-8469-4721f4664c8e-kube-api-access-6jdjl" (OuterVolumeSpecName: "kube-api-access-6jdjl") pod "66aa18ba-854e-498e-8469-4721f4664c8e" (UID: "66aa18ba-854e-498e-8469-4721f4664c8e"). InnerVolumeSpecName "kube-api-access-6jdjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:34.021114 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.021095    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311055f3-f587-4d42-a8bc-e549b2817d89-kube-api-access-mjp8c" (OuterVolumeSpecName: "kube-api-access-mjp8c") pod "311055f3-f587-4d42-a8bc-e549b2817d89" (UID: "311055f3-f587-4d42-a8bc-e549b2817d89"). InnerVolumeSpecName "kube-api-access-mjp8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:34.024997 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.024978    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-util" (OuterVolumeSpecName: "util") pod "311055f3-f587-4d42-a8bc-e549b2817d89" (UID: "311055f3-f587-4d42-a8bc-e549b2817d89"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:34.025186 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.025164    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-util" (OuterVolumeSpecName: "util") pod "66aa18ba-854e-498e-8469-4721f4664c8e" (UID: "66aa18ba-854e-498e-8469-4721f4664c8e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:34.119662 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.119603    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:34.119662 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.119634    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjp8c\" (UniqueName: \"kubernetes.io/projected/311055f3-f587-4d42-a8bc-e549b2817d89-kube-api-access-mjp8c\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:34.119662 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.119650    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-util\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:34.119662 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.119662    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66aa18ba-854e-498e-8469-4721f4664c8e-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:34.119850 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.119675    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jdjl\" (UniqueName: \"kubernetes.io/projected/66aa18ba-854e-498e-8469-4721f4664c8e-kube-api-access-6jdjl\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:34.119850 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.119688    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/311055f3-f587-4d42-a8bc-e549b2817d89-util\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:47:34.812610 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.812584    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw"
Jul 24 17:47:34.813027 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.812584    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw" event={"ID":"66aa18ba-854e-498e-8469-4721f4664c8e","Type":"ContainerDied","Data":"9403dd781285efdb71a4ceef9d21bf0c3999ef8815fa8efafa68666f0289ec29"}
Jul 24 17:47:34.813027 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.812722    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9403dd781285efdb71a4ceef9d21bf0c3999ef8815fa8efafa68666f0289ec29"
Jul 24 17:47:34.814482 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.814458    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr"
Jul 24 17:47:34.814602 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.814499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr" event={"ID":"311055f3-f587-4d42-a8bc-e549b2817d89","Type":"ContainerDied","Data":"a45dcf3b0ecf937761cebcf7f200716fcd07cd64d57dacef3b2fdba8e3f63900"}
Jul 24 17:47:34.814602 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:34.814539    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a45dcf3b0ecf937761cebcf7f200716fcd07cd64d57dacef3b2fdba8e3f63900"
Jul 24 17:47:39.491473 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491430    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"]
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491799    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897af391-380d-4470-9f9b-297c14483c2a" containerName="pull"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491812    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="897af391-380d-4470-9f9b-297c14483c2a" containerName="pull"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491823    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311055f3-f587-4d42-a8bc-e549b2817d89" containerName="pull"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491829    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="311055f3-f587-4d42-a8bc-e549b2817d89" containerName="pull"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491835    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66aa18ba-854e-498e-8469-4721f4664c8e" containerName="extract"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491842    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="66aa18ba-854e-498e-8469-4721f4664c8e" containerName="extract"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491858    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerName="util"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491863    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerName="util"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491869    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerName="extract"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491899    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerName="extract"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491906    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66aa18ba-854e-498e-8469-4721f4664c8e" containerName="pull"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491912    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="66aa18ba-854e-498e-8469-4721f4664c8e" containerName="pull"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491919    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897af391-380d-4470-9f9b-297c14483c2a" containerName="extract"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491924    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="897af391-380d-4470-9f9b-297c14483c2a" containerName="extract"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491948    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897af391-380d-4470-9f9b-297c14483c2a" containerName="util"
Jul 24 17:47:39.491940 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491954    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="897af391-380d-4470-9f9b-297c14483c2a" containerName="util"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491961    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerName="pull"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491966    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerName="pull"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491975    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311055f3-f587-4d42-a8bc-e549b2817d89" containerName="util"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491980    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="311055f3-f587-4d42-a8bc-e549b2817d89" containerName="util"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491987    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66aa18ba-854e-498e-8469-4721f4664c8e" containerName="util"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491991    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="66aa18ba-854e-498e-8469-4721f4664c8e" containerName="util"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.491997    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311055f3-f587-4d42-a8bc-e549b2817d89" containerName="extract"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.492002    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="311055f3-f587-4d42-a8bc-e549b2817d89" containerName="extract"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.492076    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="311055f3-f587-4d42-a8bc-e549b2817d89" containerName="extract"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.492089    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="897af391-380d-4470-9f9b-297c14483c2a" containerName="extract"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.492096    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="66aa18ba-854e-498e-8469-4721f4664c8e" containerName="extract"
Jul 24 17:47:39.492537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.492103    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bc9c580-1d34-4ee0-b364-173f8bc862b7" containerName="extract"
Jul 24 17:47:39.496436 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.496415    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:47:39.500070 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.500047    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m2tv2\""
Jul 24 17:47:39.513196 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.513171    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"]
Jul 24 17:47:39.561984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.561959    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bvzj\" (UniqueName: \"kubernetes.io/projected/2f801d28-346a-4236-84eb-725c066a2275-kube-api-access-5bvzj\") pod \"limitador-operator-controller-manager-859d7b78bd-4kc6d\" (UID: \"2f801d28-346a-4236-84eb-725c066a2275\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:47:39.662587 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.662555    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bvzj\" (UniqueName: \"kubernetes.io/projected/2f801d28-346a-4236-84eb-725c066a2275-kube-api-access-5bvzj\") pod \"limitador-operator-controller-manager-859d7b78bd-4kc6d\" (UID: \"2f801d28-346a-4236-84eb-725c066a2275\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:47:39.674351 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.674325    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bvzj\" (UniqueName: \"kubernetes.io/projected/2f801d28-346a-4236-84eb-725c066a2275-kube-api-access-5bvzj\") pod \"limitador-operator-controller-manager-859d7b78bd-4kc6d\" (UID: \"2f801d28-346a-4236-84eb-725c066a2275\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:47:39.806925 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.806900    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:47:39.949500 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:39.949474    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"]
Jul 24 17:47:39.950867 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:39.950839    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f801d28_346a_4236_84eb_725c066a2275.slice/crio-31ad1568fa706524e67a73e2bcf183723f69f82869043687bf422e541f619286 WatchSource:0}: Error finding container 31ad1568fa706524e67a73e2bcf183723f69f82869043687bf422e541f619286: Status 404 returned error can't find the container with id 31ad1568fa706524e67a73e2bcf183723f69f82869043687bf422e541f619286
Jul 24 17:47:40.853611 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:40.853574    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d" event={"ID":"2f801d28-346a-4236-84eb-725c066a2275","Type":"ContainerStarted","Data":"31ad1568fa706524e67a73e2bcf183723f69f82869043687bf422e541f619286"}
Jul 24 17:47:41.860822 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:41.860750    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d" event={"ID":"2f801d28-346a-4236-84eb-725c066a2275","Type":"ContainerStarted","Data":"f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f"}
Jul 24 17:47:41.861143 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:41.860903    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:47:41.878498 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:41.878453    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d" podStartSLOduration=1.236388765 podStartE2EDuration="2.878440975s" totalImagesPullingTime="1.64205221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:41.877168687 +0000 UTC m=+469.081860500" watchObservedRunningTime="2026-07-24 17:47:41.878440975 +0000 UTC m=+469.083132772"
Jul 24 17:47:43.654970 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:43.654937    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"]
Jul 24 17:47:43.658324 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:43.658308    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"
Jul 24 17:47:43.661693 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:43.661674    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-b4zqm\""
Jul 24 17:47:43.678720 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:43.678697    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"]
Jul 24 17:47:43.793730 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:43.793705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r2nm\" (UniqueName: \"kubernetes.io/projected/d8ee1ef8-7261-434a-a6ee-0a7afe24011e-kube-api-access-5r2nm\") pod \"authorino-operator-75cc8d5cb7-b5n7m\" (UID: \"d8ee1ef8-7261-434a-a6ee-0a7afe24011e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"
Jul 24 17:47:43.894020 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:43.893992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r2nm\" (UniqueName: \"kubernetes.io/projected/d8ee1ef8-7261-434a-a6ee-0a7afe24011e-kube-api-access-5r2nm\") pod \"authorino-operator-75cc8d5cb7-b5n7m\" (UID: \"d8ee1ef8-7261-434a-a6ee-0a7afe24011e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"
Jul 24 17:47:43.903001 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:43.902975    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r2nm\" (UniqueName: \"kubernetes.io/projected/d8ee1ef8-7261-434a-a6ee-0a7afe24011e-kube-api-access-5r2nm\") pod \"authorino-operator-75cc8d5cb7-b5n7m\" (UID: \"d8ee1ef8-7261-434a-a6ee-0a7afe24011e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"
Jul 24 17:47:43.968205 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:43.968148    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"
Jul 24 17:47:44.103570 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:44.103546    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"]
Jul 24 17:47:44.105440 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:44.105397    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ee1ef8_7261_434a_a6ee_0a7afe24011e.slice/crio-79874c2e9fcb619448b0c8c167275efceff6cd506ce954c1c9e8ebf7eb9c8a2b WatchSource:0}: Error finding container 79874c2e9fcb619448b0c8c167275efceff6cd506ce954c1c9e8ebf7eb9c8a2b: Status 404 returned error can't find the container with id 79874c2e9fcb619448b0c8c167275efceff6cd506ce954c1c9e8ebf7eb9c8a2b
Jul 24 17:47:44.880122 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:44.880083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m" event={"ID":"d8ee1ef8-7261-434a-a6ee-0a7afe24011e","Type":"ContainerStarted","Data":"79874c2e9fcb619448b0c8c167275efceff6cd506ce954c1c9e8ebf7eb9c8a2b"}
Jul 24 17:47:46.892449 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:46.892411    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m" event={"ID":"d8ee1ef8-7261-434a-a6ee-0a7afe24011e","Type":"ContainerStarted","Data":"96e0aea4003c9a2ca908b4fec80261dae873b75b0d738de6f4fe9b1ec1fe7f51"}
Jul 24 17:47:46.892859 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:46.892495    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"
Jul 24 17:47:46.916064 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:46.916020    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m" podStartSLOduration=1.612107237 podStartE2EDuration="3.91600507s" totalImagesPullingTime="2.303897833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:46.914288698 +0000 UTC m=+474.118980494" watchObservedRunningTime="2026-07-24 17:47:46.91600507 +0000 UTC m=+474.120696867"
Jul 24 17:47:48.559208 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.559171    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"]
Jul 24 17:47:48.562778 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.562758    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:47:48.566218 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.566201    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-g9nm2\""
Jul 24 17:47:48.581841 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.581812    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"]
Jul 24 17:47:48.631491 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.631467    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq492\" (UniqueName: \"kubernetes.io/projected/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-kube-api-access-vq492\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" (UID: \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:47:48.631576 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.631500    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" (UID: \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:47:48.732445 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.732420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" (UID: \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:47:48.732537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.732510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq492\" (UniqueName: \"kubernetes.io/projected/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-kube-api-access-vq492\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" (UID: \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:47:48.732840 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.732823    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" (UID: \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:47:48.741612 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.741588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq492\" (UniqueName: \"kubernetes.io/projected/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-kube-api-access-vq492\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" (UID: \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:47:48.873342 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:48.873288    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:47:49.008022 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:49.007950    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"]
Jul 24 17:47:49.009288 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:49.009257    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3239d5a5_2ec5_43df_9c28_1e61d62ec72c.slice/crio-ab452aab084246c4eb002512f16a55ae840673500b4574a393e2cbb13c8251bf WatchSource:0}: Error finding container ab452aab084246c4eb002512f16a55ae840673500b4574a393e2cbb13c8251bf: Status 404 returned error can't find the container with id ab452aab084246c4eb002512f16a55ae840673500b4574a393e2cbb13c8251bf
Jul 24 17:47:49.912039 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:49.912008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k" event={"ID":"3239d5a5-2ec5-43df-9c28-1e61d62ec72c","Type":"ContainerStarted","Data":"ab452aab084246c4eb002512f16a55ae840673500b4574a393e2cbb13c8251bf"}
Jul 24 17:47:51.663515 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.663483    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"]
Jul 24 17:47:51.672011 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.671986    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"
Jul 24 17:47:51.675207 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.675180    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7sl9l\""
Jul 24 17:47:51.675300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.675257    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 24 17:47:51.689550 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.689529    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"]
Jul 24 17:47:51.759511 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.759485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ztrr\" (UniqueName: \"kubernetes.io/projected/f43329ad-5912-4f77-9177-e70f7089b65e-kube-api-access-5ztrr\") pod \"dns-operator-controller-manager-864dd78c4c-lnpmv\" (UID: \"f43329ad-5912-4f77-9177-e70f7089b65e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"
Jul 24 17:47:51.860218 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.860187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ztrr\" (UniqueName: \"kubernetes.io/projected/f43329ad-5912-4f77-9177-e70f7089b65e-kube-api-access-5ztrr\") pod \"dns-operator-controller-manager-864dd78c4c-lnpmv\" (UID: \"f43329ad-5912-4f77-9177-e70f7089b65e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"
Jul 24 17:47:51.869390 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.869369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ztrr\" (UniqueName: \"kubernetes.io/projected/f43329ad-5912-4f77-9177-e70f7089b65e-kube-api-access-5ztrr\") pod \"dns-operator-controller-manager-864dd78c4c-lnpmv\" (UID: \"f43329ad-5912-4f77-9177-e70f7089b65e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"
Jul 24 17:47:51.965790 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.965721    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f5bf9689b-ztkgv"]
Jul 24 17:47:51.970263 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.970241    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:51.983308 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.983282    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f5bf9689b-ztkgv"]
Jul 24 17:47:51.985006 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:51.984964    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"
Jul 24 17:47:52.062182 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.062139    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-service-ca\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.062352 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.062228    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-console-config\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.062352 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.062257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-oauth-serving-cert\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.062469 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.062400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m95d8\" (UniqueName: \"kubernetes.io/projected/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-kube-api-access-m95d8\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.062469 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.062434    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-trusted-ca-bundle\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.062583 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.062474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-console-oauth-config\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.062583 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.062502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-console-serving-cert\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.131082 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.131057    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"]
Jul 24 17:47:52.132460 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:52.132433    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43329ad_5912_4f77_9177_e70f7089b65e.slice/crio-2c37f9fc719246be19ccc33cd0036e56bf6cedfa6ea8d4eecdc693cdb08daf75 WatchSource:0}: Error finding container 2c37f9fc719246be19ccc33cd0036e56bf6cedfa6ea8d4eecdc693cdb08daf75: Status 404 returned error can't find the container with id 2c37f9fc719246be19ccc33cd0036e56bf6cedfa6ea8d4eecdc693cdb08daf75
Jul 24 17:47:52.163708 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.163682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m95d8\" (UniqueName: \"kubernetes.io/projected/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-kube-api-access-m95d8\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.163813 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.163712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-trusted-ca-bundle\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.163813 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.163736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-console-oauth-config\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.163948 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.163846    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-console-serving-cert\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.163948 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.163912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-service-ca\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.164051 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.163963    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-console-config\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.164051 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.163981    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-oauth-serving-cert\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.164562 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.164543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-trusted-ca-bundle\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.164727 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.164705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-oauth-serving-cert\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.164791 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.164706    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-service-ca\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.164791 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.164746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-console-config\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.166316 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.166290    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-console-oauth-config\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.166398 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.166351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-console-serving-cert\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.177054 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.177031    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m95d8\" (UniqueName: \"kubernetes.io/projected/a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38-kube-api-access-m95d8\") pod \"console-f5bf9689b-ztkgv\" (UID: \"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38\") " pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.283407 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.283331    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:47:52.869741 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.869709    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:47:52.940422 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:52.940346    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv" event={"ID":"f43329ad-5912-4f77-9177-e70f7089b65e","Type":"ContainerStarted","Data":"2c37f9fc719246be19ccc33cd0036e56bf6cedfa6ea8d4eecdc693cdb08daf75"}
Jul 24 17:47:53.509905 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:53.509859    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f5bf9689b-ztkgv"]
Jul 24 17:47:53.511320 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:53.511296    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1f02c7e_e4b1_4cb1_bc4c_6117993c9d38.slice/crio-c0a06534df54928029a1ad5d4a2887bbcd935dbfff7b31106e569a43ff977f5d WatchSource:0}: Error finding container c0a06534df54928029a1ad5d4a2887bbcd935dbfff7b31106e569a43ff977f5d: Status 404 returned error can't find the container with id c0a06534df54928029a1ad5d4a2887bbcd935dbfff7b31106e569a43ff977f5d
Jul 24 17:47:53.950697 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:53.950656    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k" event={"ID":"3239d5a5-2ec5-43df-9c28-1e61d62ec72c","Type":"ContainerStarted","Data":"6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d"}
Jul 24 17:47:53.951092 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:53.951034    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:47:53.952934 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:53.952905    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5bf9689b-ztkgv" event={"ID":"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38","Type":"ContainerStarted","Data":"d3fd0a7853c3de46a2eeba62d16714a7ef7608d99e2603aceb8fe0d6246419ee"}
Jul 24 17:47:53.953049 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:53.952953    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5bf9689b-ztkgv" event={"ID":"a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38","Type":"ContainerStarted","Data":"c0a06534df54928029a1ad5d4a2887bbcd935dbfff7b31106e569a43ff977f5d"}
Jul 24 17:47:53.971559 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:53.971511    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k" podStartSLOduration=1.55505779 podStartE2EDuration="5.971497433s" totalImagesPullingTime="4.416439643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:53.969953672 +0000 UTC m=+481.174645468" watchObservedRunningTime="2026-07-24 17:47:53.971497433 +0000 UTC m=+481.176189229"
Jul 24 17:47:53.988483 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:53.988435    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f5bf9689b-ztkgv" podStartSLOduration=2.988420457 podStartE2EDuration="2.988420457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:53.986130912 +0000 UTC m=+481.190822710" watchObservedRunningTime="2026-07-24 17:47:53.988420457 +0000 UTC m=+481.193112255"
Jul 24 17:47:54.780833 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:54.780811    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 24 17:47:54.960701 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:54.960667    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv" event={"ID":"f43329ad-5912-4f77-9177-e70f7089b65e","Type":"ContainerStarted","Data":"7dddd7a772e6d5a74bbe965a4ac6069edf39fb9ce87202c3316b945ed18e7df8"}
Jul 24 17:47:54.961152 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:54.960901    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"
Jul 24 17:47:54.980484 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:54.980444    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv" podStartSLOduration=1.337014085 podStartE2EDuration="3.980431112s" totalImagesPullingTime="2.643417027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:54.978292886 +0000 UTC m=+482.182984685" watchObservedRunningTime="2026-07-24 17:47:54.980431112 +0000 UTC m=+482.185122909"
Jul 24 17:47:56.290638 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.290601    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"]
Jul 24 17:47:56.294225 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.294201    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.296686 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.296663    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 24 17:47:56.296787 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.296701    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 24 17:47:56.296787 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.296715    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gd6n5\""
Jul 24 17:47:56.300772 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.300746    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"]
Jul 24 17:47:56.402159 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.402134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/452d88e5-953a-4211-8a5f-2eef3c1c7a8a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rsxj7\" (UID: \"452d88e5-953a-4211-8a5f-2eef3c1c7a8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.402274 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.402174    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/452d88e5-953a-4211-8a5f-2eef3c1c7a8a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rsxj7\" (UID: \"452d88e5-953a-4211-8a5f-2eef3c1c7a8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.402274 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.402260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlp82\" (UniqueName: \"kubernetes.io/projected/452d88e5-953a-4211-8a5f-2eef3c1c7a8a-kube-api-access-nlp82\") pod \"kuadrant-console-plugin-66b5774985-rsxj7\" (UID: \"452d88e5-953a-4211-8a5f-2eef3c1c7a8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.503375 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.503348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/452d88e5-953a-4211-8a5f-2eef3c1c7a8a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rsxj7\" (UID: \"452d88e5-953a-4211-8a5f-2eef3c1c7a8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.503493 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.503388    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/452d88e5-953a-4211-8a5f-2eef3c1c7a8a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rsxj7\" (UID: \"452d88e5-953a-4211-8a5f-2eef3c1c7a8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.503493 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.503439    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlp82\" (UniqueName: \"kubernetes.io/projected/452d88e5-953a-4211-8a5f-2eef3c1c7a8a-kube-api-access-nlp82\") pod \"kuadrant-console-plugin-66b5774985-rsxj7\" (UID: \"452d88e5-953a-4211-8a5f-2eef3c1c7a8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.504027 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.504005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/452d88e5-953a-4211-8a5f-2eef3c1c7a8a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rsxj7\" (UID: \"452d88e5-953a-4211-8a5f-2eef3c1c7a8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.505862 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.505843    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/452d88e5-953a-4211-8a5f-2eef3c1c7a8a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rsxj7\" (UID: \"452d88e5-953a-4211-8a5f-2eef3c1c7a8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.511473 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.511448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlp82\" (UniqueName: \"kubernetes.io/projected/452d88e5-953a-4211-8a5f-2eef3c1c7a8a-kube-api-access-nlp82\") pod \"kuadrant-console-plugin-66b5774985-rsxj7\" (UID: \"452d88e5-953a-4211-8a5f-2eef3c1c7a8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.604875 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.604829    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"
Jul 24 17:47:56.738007 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.737980    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7"]
Jul 24 17:47:56.739280 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:47:56.739249    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod452d88e5_953a_4211_8a5f_2eef3c1c7a8a.slice/crio-595426035c6f5b430d931e63cd1cf059f68629178defa5d22520fa5887d4e231 WatchSource:0}: Error finding container 595426035c6f5b430d931e63cd1cf059f68629178defa5d22520fa5887d4e231: Status 404 returned error can't find the container with id 595426035c6f5b430d931e63cd1cf059f68629178defa5d22520fa5887d4e231
Jul 24 17:47:56.973950 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:56.973870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7" event={"ID":"452d88e5-953a-4211-8a5f-2eef3c1c7a8a","Type":"ContainerStarted","Data":"595426035c6f5b430d931e63cd1cf059f68629178defa5d22520fa5887d4e231"}
Jul 24 17:47:57.901013 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:47:57.900982    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b5n7m"
Jul 24 17:48:02.283539 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:02.283500    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:48:02.284041 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:02.283598    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:48:02.289215 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:02.289193    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:48:03.020221 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:03.020192    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f5bf9689b-ztkgv"
Jul 24 17:48:03.067684 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:03.067647    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f979b4bcf-wvrcq"]
Jul 24 17:48:04.963700 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:04.963667    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:48:05.969533 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:05.969498    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lnpmv"
Jul 24 17:48:06.790381 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.790347    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"]
Jul 24 17:48:06.790604 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.790574    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" containerName="manager" containerID="cri-o://6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d" gracePeriod=2
Jul 24 17:48:06.806734 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.806709    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"]
Jul 24 17:48:06.813345 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:06.813296    2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-mf92k\": pods \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:48:06.818977 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.818382    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"]
Jul 24 17:48:06.819095 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.819023    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" containerName="manager"
Jul 24 17:48:06.819095 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.819048    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" containerName="manager"
Jul 24 17:48:06.819495 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.819219    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" containerName="manager"
Jul 24 17:48:06.837319 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.836747    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"]
Jul 24 17:48:06.837319 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.836794    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"]
Jul 24 17:48:06.837319 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.836987    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d" podUID="2f801d28-346a-4236-84eb-725c066a2275" containerName="manager" containerID="cri-o://f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f" gracePeriod=2
Jul 24 17:48:06.837319 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.837193    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:06.840042 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:06.840018    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:48:06.842859 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:06.842831    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4kc6d\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="2f801d28-346a-4236-84eb-725c066a2275" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:48:06.846010 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.845968    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"]
Jul 24 17:48:06.865963 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.864726    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"]
Jul 24 17:48:06.865963 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.865525    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f801d28-346a-4236-84eb-725c066a2275" containerName="manager"
Jul 24 17:48:06.865963 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.865545    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f801d28-346a-4236-84eb-725c066a2275" containerName="manager"
Jul 24 17:48:06.865963 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.865762    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f801d28-346a-4236-84eb-725c066a2275" containerName="manager"
Jul 24 17:48:06.871283 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.871147    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"
Jul 24 17:48:06.875963 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:06.875053    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:48:06.877678 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.877653    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"]
Jul 24 17:48:06.896415 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:06.896389    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4kc6d\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="2f801d28-346a-4236-84eb-725c066a2275" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:48:06.997843 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.997815    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s9tf\" (UniqueName: \"kubernetes.io/projected/cb8775b1-dd55-472b-824a-2fefb04a9fac-kube-api-access-9s9tf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nhqm5\" (UID: \"cb8775b1-dd55-472b-824a-2fefb04a9fac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:06.998190 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.997859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb48m\" (UniqueName: \"kubernetes.io/projected/11407094-b4c8-4d0b-a597-e11f04ef6ec1-kube-api-access-fb48m\") pod \"limitador-operator-controller-manager-859d7b78bd-x7rw4\" (UID: \"11407094-b4c8-4d0b-a597-e11f04ef6ec1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"
Jul 24 17:48:06.998190 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:06.997916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb8775b1-dd55-472b-824a-2fefb04a9fac-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nhqm5\" (UID: \"cb8775b1-dd55-472b-824a-2fefb04a9fac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:07.099067 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:07.098985    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s9tf\" (UniqueName: \"kubernetes.io/projected/cb8775b1-dd55-472b-824a-2fefb04a9fac-kube-api-access-9s9tf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nhqm5\" (UID: \"cb8775b1-dd55-472b-824a-2fefb04a9fac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:07.099067 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:07.099046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb48m\" (UniqueName: \"kubernetes.io/projected/11407094-b4c8-4d0b-a597-e11f04ef6ec1-kube-api-access-fb48m\") pod \"limitador-operator-controller-manager-859d7b78bd-x7rw4\" (UID: \"11407094-b4c8-4d0b-a597-e11f04ef6ec1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"
Jul 24 17:48:07.099272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:07.099081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb8775b1-dd55-472b-824a-2fefb04a9fac-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nhqm5\" (UID: \"cb8775b1-dd55-472b-824a-2fefb04a9fac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:07.099471 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:07.099426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb8775b1-dd55-472b-824a-2fefb04a9fac-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nhqm5\" (UID: \"cb8775b1-dd55-472b-824a-2fefb04a9fac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:07.109559 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:07.109538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb48m\" (UniqueName: \"kubernetes.io/projected/11407094-b4c8-4d0b-a597-e11f04ef6ec1-kube-api-access-fb48m\") pod \"limitador-operator-controller-manager-859d7b78bd-x7rw4\" (UID: \"11407094-b4c8-4d0b-a597-e11f04ef6ec1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"
Jul 24 17:48:07.109910 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:07.109872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s9tf\" (UniqueName: \"kubernetes.io/projected/cb8775b1-dd55-472b-824a-2fefb04a9fac-kube-api-access-9s9tf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nhqm5\" (UID: \"cb8775b1-dd55-472b-824a-2fefb04a9fac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:07.247313 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:07.247061    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:07.254731 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:07.254705    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"
Jul 24 17:48:17.615943 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.615914    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:48:17.618678 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:17.618648    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4kc6d\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="2f801d28-346a-4236-84eb-725c066a2275" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:48:17.637235 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.637060    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:48:17.639592 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:17.639434    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:48:17.641700 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:17.641665    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4kc6d\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="2f801d28-346a-4236-84eb-725c066a2275" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:48:17.670865 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.670830    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"]
Jul 24 17:48:17.671314 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:48:17.671289    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb8775b1_dd55_472b_824a_2fefb04a9fac.slice/crio-dcd7822f193698e692b6a00adc1c84a1f0e2f8f2b0a3240e6259e367820b45be WatchSource:0}: Error finding container dcd7822f193698e692b6a00adc1c84a1f0e2f8f2b0a3240e6259e367820b45be: Status 404 returned error can't find the container with id dcd7822f193698e692b6a00adc1c84a1f0e2f8f2b0a3240e6259e367820b45be
Jul 24 17:48:17.692164 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.692146    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"]
Jul 24 17:48:17.693722 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:48:17.693702    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11407094_b4c8_4d0b_a597_e11f04ef6ec1.slice/crio-3e899963e50f5e3760b61850db6253b66ebad2df814b9d6f60538890a9110e4e WatchSource:0}: Error finding container 3e899963e50f5e3760b61850db6253b66ebad2df814b9d6f60538890a9110e4e: Status 404 returned error can't find the container with id 3e899963e50f5e3760b61850db6253b66ebad2df814b9d6f60538890a9110e4e
Jul 24 17:48:17.700415 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.700398    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bvzj\" (UniqueName: \"kubernetes.io/projected/2f801d28-346a-4236-84eb-725c066a2275-kube-api-access-5bvzj\") pod \"2f801d28-346a-4236-84eb-725c066a2275\" (UID: \"2f801d28-346a-4236-84eb-725c066a2275\") "
Jul 24 17:48:17.700462 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.700439    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq492\" (UniqueName: \"kubernetes.io/projected/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-kube-api-access-vq492\") pod \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\" (UID: \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\") "
Jul 24 17:48:17.700508 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.700464    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-extensions-socket-volume\") pod \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\" (UID: \"3239d5a5-2ec5-43df-9c28-1e61d62ec72c\") "
Jul 24 17:48:17.700976 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.700945    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3239d5a5-2ec5-43df-9c28-1e61d62ec72c" (UID: "3239d5a5-2ec5-43df-9c28-1e61d62ec72c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:17.702497 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.702467    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f801d28-346a-4236-84eb-725c066a2275-kube-api-access-5bvzj" (OuterVolumeSpecName: "kube-api-access-5bvzj") pod "2f801d28-346a-4236-84eb-725c066a2275" (UID: "2f801d28-346a-4236-84eb-725c066a2275"). InnerVolumeSpecName "kube-api-access-5bvzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:17.702575 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.702534    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-kube-api-access-vq492" (OuterVolumeSpecName: "kube-api-access-vq492") pod "3239d5a5-2ec5-43df-9c28-1e61d62ec72c" (UID: "3239d5a5-2ec5-43df-9c28-1e61d62ec72c"). InnerVolumeSpecName "kube-api-access-vq492". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:17.801340 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.801319    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bvzj\" (UniqueName: \"kubernetes.io/projected/2f801d28-346a-4236-84eb-725c066a2275-kube-api-access-5bvzj\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:17.801340 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.801339    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq492\" (UniqueName: \"kubernetes.io/projected/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-kube-api-access-vq492\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:17.801488 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:17.801349    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3239d5a5-2ec5-43df-9c28-1e61d62ec72c-extensions-socket-volume\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:18.118831 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.118792    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5" event={"ID":"cb8775b1-dd55-472b-824a-2fefb04a9fac","Type":"ContainerStarted","Data":"5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755"}
Jul 24 17:48:18.119044 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.118908    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5" event={"ID":"cb8775b1-dd55-472b-824a-2fefb04a9fac","Type":"ContainerStarted","Data":"dcd7822f193698e692b6a00adc1c84a1f0e2f8f2b0a3240e6259e367820b45be"}
Jul 24 17:48:18.119044 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.118939    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:18.120401 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.120371    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7" event={"ID":"452d88e5-953a-4211-8a5f-2eef3c1c7a8a","Type":"ContainerStarted","Data":"263139f6697b6e8bcb4cf3eb646d8e836c625b9ccc1859a691ae77061271e693"}
Jul 24 17:48:18.121054 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:18.121028    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:48:18.121678 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.121657    2562 generic.go:356] "Generic (PLEG): container finished" podID="2f801d28-346a-4236-84eb-725c066a2275" containerID="f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f" exitCode=0
Jul 24 17:48:18.121792 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.121701    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:48:18.121792 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.121734    2562 scope.go:117] "RemoveContainer" containerID="f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f"
Jul 24 17:48:18.123012 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:18.122990    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4kc6d\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="2f801d28-346a-4236-84eb-725c066a2275" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:48:18.123196 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.123166    2562 generic.go:356] "Generic (PLEG): container finished" podID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" containerID="6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d" exitCode=0
Jul 24 17:48:18.123298 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.123212    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:48:18.125169 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.125146    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4" event={"ID":"11407094-b4c8-4d0b-a597-e11f04ef6ec1","Type":"ContainerStarted","Data":"1737ed208dc9c44beb0a77f9f688b008cd025fec4f84980b1c52313b607f3115"}
Jul 24 17:48:18.125272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.125191    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4" event={"ID":"11407094-b4c8-4d0b-a597-e11f04ef6ec1","Type":"ContainerStarted","Data":"3e899963e50f5e3760b61850db6253b66ebad2df814b9d6f60538890a9110e4e"}
Jul 24 17:48:18.125272 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.125238    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"
Jul 24 17:48:18.131578 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.131560    2562 scope.go:117] "RemoveContainer" containerID="f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f"
Jul 24 17:48:18.131836 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:18.131818    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f\": container with ID starting with f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f not found: ID does not exist" containerID="f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f"
Jul 24 17:48:18.131916 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.131842    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f"} err="failed to get container status \"f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f\": rpc error: code = NotFound desc = could not find container \"f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f\": container with ID starting with f33ae84811803ef5cb91e5f678eb2994a46440e5fe023e716b38126bb7b5287f not found: ID does not exist"
Jul 24 17:48:18.131916 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.131857    2562 scope.go:117] "RemoveContainer" containerID="6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d"
Jul 24 17:48:18.140580 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.140563    2562 scope.go:117] "RemoveContainer" containerID="6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d"
Jul 24 17:48:18.140815 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:18.140799    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d\": container with ID starting with 6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d not found: ID does not exist" containerID="6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d"
Jul 24 17:48:18.140858 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.140820    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d"} err="failed to get container status \"6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d\": rpc error: code = NotFound desc = could not find container \"6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d\": container with ID starting with 6ab06aa6806b21a76457d8859372c1eb7713b97dc896ab7f7746e4f36c26aa3d not found: ID does not exist"
Jul 24 17:48:18.148172 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.148136    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5" podStartSLOduration=12.148125215 podStartE2EDuration="12.148125215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:18.145827634 +0000 UTC m=+505.350519430" watchObservedRunningTime="2026-07-24 17:48:18.148125215 +0000 UTC m=+505.352817084"
Jul 24 17:48:18.148342 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:18.148320    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mf92k\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mf92k"
Jul 24 17:48:18.168004 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.167969    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rsxj7" podStartSLOduration=1.378425378 podStartE2EDuration="22.167957918s" totalImagesPullingTime="20.78953254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:18.167290931 +0000 UTC m=+505.371982728" watchObservedRunningTime="2026-07-24 17:48:18.167957918 +0000 UTC m=+505.372649715"
Jul 24 17:48:18.182584 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:18.182548    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4" podStartSLOduration=12.182536589 podStartE2EDuration="12.182536589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:18.182367705 +0000 UTC m=+505.387059502" watchObservedRunningTime="2026-07-24 17:48:18.182536589 +0000 UTC m=+505.387228386"
Jul 24 17:48:18.184623 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:18.184594    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4kc6d\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="2f801d28-346a-4236-84eb-725c066a2275" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4kc6d"
Jul 24 17:48:19.409596 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:19.409552    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f801d28-346a-4236-84eb-725c066a2275" path="/var/lib/kubelet/pods/2f801d28-346a-4236-84eb-725c066a2275/volumes"
Jul 24 17:48:19.410029 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:19.409951    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3239d5a5-2ec5-43df-9c28-1e61d62ec72c" path="/var/lib/kubelet/pods/3239d5a5-2ec5-43df-9c28-1e61d62ec72c/volumes"
Jul 24 17:48:22.539385 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.539354    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"]
Jul 24 17:48:22.539842 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.539563    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5" podUID="cb8775b1-dd55-472b-824a-2fefb04a9fac" containerName="manager" containerID="cri-o://5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755" gracePeriod=10
Jul 24 17:48:22.541279 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.541252    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:22.799014 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.798954    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:22.834440 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.834415    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb8775b1-dd55-472b-824a-2fefb04a9fac-extensions-socket-volume\") pod \"cb8775b1-dd55-472b-824a-2fefb04a9fac\" (UID: \"cb8775b1-dd55-472b-824a-2fefb04a9fac\") "
Jul 24 17:48:22.834604 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.834452    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9s9tf\" (UniqueName: \"kubernetes.io/projected/cb8775b1-dd55-472b-824a-2fefb04a9fac-kube-api-access-9s9tf\") pod \"cb8775b1-dd55-472b-824a-2fefb04a9fac\" (UID: \"cb8775b1-dd55-472b-824a-2fefb04a9fac\") "
Jul 24 17:48:22.834839 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.834803    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb8775b1-dd55-472b-824a-2fefb04a9fac-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cb8775b1-dd55-472b-824a-2fefb04a9fac" (UID: "cb8775b1-dd55-472b-824a-2fefb04a9fac"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:22.836632 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.836604    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb8775b1-dd55-472b-824a-2fefb04a9fac-kube-api-access-9s9tf" (OuterVolumeSpecName: "kube-api-access-9s9tf") pod "cb8775b1-dd55-472b-824a-2fefb04a9fac" (UID: "cb8775b1-dd55-472b-824a-2fefb04a9fac"). InnerVolumeSpecName "kube-api-access-9s9tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:22.935237 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.935212    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb8775b1-dd55-472b-824a-2fefb04a9fac-extensions-socket-volume\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:22.935237 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:22.935236    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s9tf\" (UniqueName: \"kubernetes.io/projected/cb8775b1-dd55-472b-824a-2fefb04a9fac-kube-api-access-9s9tf\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:23.160172 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.160105    2562 generic.go:356] "Generic (PLEG): container finished" podID="cb8775b1-dd55-472b-824a-2fefb04a9fac" containerID="5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755" exitCode=0
Jul 24 17:48:23.160172 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.160159    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"
Jul 24 17:48:23.160295 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.160178    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5" event={"ID":"cb8775b1-dd55-472b-824a-2fefb04a9fac","Type":"ContainerDied","Data":"5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755"}
Jul 24 17:48:23.160295 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.160260    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5" event={"ID":"cb8775b1-dd55-472b-824a-2fefb04a9fac","Type":"ContainerDied","Data":"dcd7822f193698e692b6a00adc1c84a1f0e2f8f2b0a3240e6259e367820b45be"}
Jul 24 17:48:23.160295 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.160288    2562 scope.go:117] "RemoveContainer" containerID="5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755"
Jul 24 17:48:23.170593 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.170571    2562 scope.go:117] "RemoveContainer" containerID="5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755"
Jul 24 17:48:23.170915 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:23.170857    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755\": container with ID starting with 5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755 not found: ID does not exist" containerID="5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755"
Jul 24 17:48:23.170915 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.170908    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755"} err="failed to get container status \"5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755\": rpc error: code = NotFound desc = could not find container \"5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755\": container with ID starting with 5190a27e1f1b7d2dc45dcc7870b67769f4ac2b9a7b9dd722f10a3ac54a832755 not found: ID does not exist"
Jul 24 17:48:23.187935 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.187914    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"]
Jul 24 17:48:23.193434 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.193407    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nhqm5"]
Jul 24 17:48:23.407057 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:23.407034    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb8775b1-dd55-472b-824a-2fefb04a9fac" path="/var/lib/kubelet/pods/cb8775b1-dd55-472b-824a-2fefb04a9fac/volumes"
Jul 24 17:48:28.096277 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.096239    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f979b4bcf-wvrcq" podUID="28ef342e-90d9-46f8-82f6-1eb953707974" containerName="console" containerID="cri-o://6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384" gracePeriod=15
Jul 24 17:48:28.348718 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.348663    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:48:28.374251 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374224    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-console-config\") pod \"28ef342e-90d9-46f8-82f6-1eb953707974\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") "
Jul 24 17:48:28.374378 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374280    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-oauth-config\") pod \"28ef342e-90d9-46f8-82f6-1eb953707974\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") "
Jul 24 17:48:28.374378 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374336    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mx2bg\" (UniqueName: \"kubernetes.io/projected/28ef342e-90d9-46f8-82f6-1eb953707974-kube-api-access-mx2bg\") pod \"28ef342e-90d9-46f8-82f6-1eb953707974\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") "
Jul 24 17:48:28.374494 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374381    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-service-ca\") pod \"28ef342e-90d9-46f8-82f6-1eb953707974\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") "
Jul 24 17:48:28.374494 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374411    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-serving-cert\") pod \"28ef342e-90d9-46f8-82f6-1eb953707974\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") "
Jul 24 17:48:28.374494 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374444    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-oauth-serving-cert\") pod \"28ef342e-90d9-46f8-82f6-1eb953707974\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") "
Jul 24 17:48:28.374494 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374472    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-trusted-ca-bundle\") pod \"28ef342e-90d9-46f8-82f6-1eb953707974\" (UID: \"28ef342e-90d9-46f8-82f6-1eb953707974\") "
Jul 24 17:48:28.374754 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374582    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-console-config" (OuterVolumeSpecName: "console-config") pod "28ef342e-90d9-46f8-82f6-1eb953707974" (UID: "28ef342e-90d9-46f8-82f6-1eb953707974"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:28.374812 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374780    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-service-ca" (OuterVolumeSpecName: "service-ca") pod "28ef342e-90d9-46f8-82f6-1eb953707974" (UID: "28ef342e-90d9-46f8-82f6-1eb953707974"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:28.374922 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.374787    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-console-config\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:28.375046 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.375008    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "28ef342e-90d9-46f8-82f6-1eb953707974" (UID: "28ef342e-90d9-46f8-82f6-1eb953707974"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:28.375127 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.375089    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "28ef342e-90d9-46f8-82f6-1eb953707974" (UID: "28ef342e-90d9-46f8-82f6-1eb953707974"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:28.377021 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.376954    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "28ef342e-90d9-46f8-82f6-1eb953707974" (UID: "28ef342e-90d9-46f8-82f6-1eb953707974"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:48:28.377021 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.376982    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "28ef342e-90d9-46f8-82f6-1eb953707974" (UID: "28ef342e-90d9-46f8-82f6-1eb953707974"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:48:28.377151 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.377048    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28ef342e-90d9-46f8-82f6-1eb953707974-kube-api-access-mx2bg" (OuterVolumeSpecName: "kube-api-access-mx2bg") pod "28ef342e-90d9-46f8-82f6-1eb953707974" (UID: "28ef342e-90d9-46f8-82f6-1eb953707974"). InnerVolumeSpecName "kube-api-access-mx2bg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:28.475435 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.475344    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx2bg\" (UniqueName: \"kubernetes.io/projected/28ef342e-90d9-46f8-82f6-1eb953707974-kube-api-access-mx2bg\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:28.475435 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.475370    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:28.475435 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.475385    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-serving-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:28.475435 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.475401    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-oauth-serving-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:28.475435 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.475415    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef342e-90d9-46f8-82f6-1eb953707974-trusted-ca-bundle\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:28.475435 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:28.475437    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28ef342e-90d9-46f8-82f6-1eb953707974-console-oauth-config\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:48:29.137226 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.137193    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x7rw4"
Jul 24 17:48:29.195551 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.195520    2562 generic.go:356] "Generic (PLEG): container finished" podID="28ef342e-90d9-46f8-82f6-1eb953707974" containerID="6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384" exitCode=0
Jul 24 17:48:29.195682 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.195561    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f979b4bcf-wvrcq"
Jul 24 17:48:29.195682 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.195597    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f979b4bcf-wvrcq" event={"ID":"28ef342e-90d9-46f8-82f6-1eb953707974","Type":"ContainerDied","Data":"6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384"}
Jul 24 17:48:29.195682 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.195643    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f979b4bcf-wvrcq" event={"ID":"28ef342e-90d9-46f8-82f6-1eb953707974","Type":"ContainerDied","Data":"3b6c89273f06106257e2c856c47272a721f45c6d82ca5d088974a9429d0d6368"}
Jul 24 17:48:29.195682 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.195664    2562 scope.go:117] "RemoveContainer" containerID="6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384"
Jul 24 17:48:29.205753 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.205735    2562 scope.go:117] "RemoveContainer" containerID="6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384"
Jul 24 17:48:29.206351 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:48:29.206046    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384\": container with ID starting with 6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384 not found: ID does not exist" containerID="6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384"
Jul 24 17:48:29.206351 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.206076    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384"} err="failed to get container status \"6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384\": rpc error: code = NotFound desc = could not find container \"6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384\": container with ID starting with 6fdcaab03baed64dc4426bf0400ed304a0c760d503cda148bde406f4ceb1d384 not found: ID does not exist"
Jul 24 17:48:29.222254 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.222225    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f979b4bcf-wvrcq"]
Jul 24 17:48:29.226420 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.226394    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f979b4bcf-wvrcq"]
Jul 24 17:48:29.407304 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:29.407251    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ef342e-90d9-46f8-82f6-1eb953707974" path="/var/lib/kubelet/pods/28ef342e-90d9-46f8-82f6-1eb953707974/volumes"
Jul 24 17:48:38.988052 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:38.988021    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"]
Jul 24 17:48:38.988511 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:38.988387    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ef342e-90d9-46f8-82f6-1eb953707974" containerName="console"
Jul 24 17:48:38.988511 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:38.988397    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ef342e-90d9-46f8-82f6-1eb953707974" containerName="console"
Jul 24 17:48:38.988511 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:38.988408    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb8775b1-dd55-472b-824a-2fefb04a9fac" containerName="manager"
Jul 24 17:48:38.988511 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:38.988413    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb8775b1-dd55-472b-824a-2fefb04a9fac" containerName="manager"
Jul 24 17:48:38.988511 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:38.988485    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ef342e-90d9-46f8-82f6-1eb953707974" containerName="console"
Jul 24 17:48:38.988511 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:38.988493    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb8775b1-dd55-472b-824a-2fefb04a9fac" containerName="manager"
Jul 24 17:48:39.163753 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.163725    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"]
Jul 24 17:48:39.163933 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.163825    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.166337 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.166312    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-lc2nh\""
Jul 24 17:48:39.166474 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.166315    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 17:48:39.252543 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.252485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/391b8927-4198-458a-92b4-e3528d751abe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.252543 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.252513    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/391b8927-4198-458a-92b4-e3528d751abe-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.252543 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.252538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.252733 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.252572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.252733 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.252591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.252733 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.252631    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.252733 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.252660    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsskg\" (UniqueName: \"kubernetes.io/projected/391b8927-4198-458a-92b4-e3528d751abe-kube-api-access-fsskg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.252733 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.252682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.252733 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.252707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/391b8927-4198-458a-92b4-e3528d751abe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.353716 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.353686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.353853 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.353728    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.353853 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.353757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.353853 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.353790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.353853 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.353819    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsskg\" (UniqueName: \"kubernetes.io/projected/391b8927-4198-458a-92b4-e3528d751abe-kube-api-access-fsskg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.353853 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.353848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.354136 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.353937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/391b8927-4198-458a-92b4-e3528d751abe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.354136 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.354005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/391b8927-4198-458a-92b4-e3528d751abe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.354136 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.354037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/391b8927-4198-458a-92b4-e3528d751abe-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.354439 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.354419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.354534 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.354514    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.354672 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.354635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.354672 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.354639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.355292 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.355265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/391b8927-4198-458a-92b4-e3528d751abe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.362517 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.362484    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/391b8927-4198-458a-92b4-e3528d751abe-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.364590 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.364564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/391b8927-4198-458a-92b4-e3528d751abe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.364992 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.364973    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsskg\" (UniqueName: \"kubernetes.io/projected/391b8927-4198-458a-92b4-e3528d751abe-kube-api-access-fsskg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.365312 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.365292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/391b8927-4198-458a-92b4-e3528d751abe-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qbgk5\" (UID: \"391b8927-4198-458a-92b4-e3528d751abe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.474561 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.474534    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:39.649533 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:39.649506    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"]
Jul 24 17:48:39.652246 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:48:39.652224    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod391b8927_4198_458a_92b4_e3528d751abe.slice/crio-698a7df12f0f8a18ebc52803b7c46a1b7768724ab7852e4752a8b699b491a266 WatchSource:0}: Error finding container 698a7df12f0f8a18ebc52803b7c46a1b7768724ab7852e4752a8b699b491a266: Status 404 returned error can't find the container with id 698a7df12f0f8a18ebc52803b7c46a1b7768724ab7852e4752a8b699b491a266
Jul 24 17:48:40.259968 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:40.259925    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" event={"ID":"391b8927-4198-458a-92b4-e3528d751abe","Type":"ContainerStarted","Data":"698a7df12f0f8a18ebc52803b7c46a1b7768724ab7852e4752a8b699b491a266"}
Jul 24 17:48:43.122149 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:43.122108    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:48:43.122635 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:43.122199    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:48:43.122635 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:43.122247    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:48:43.284220 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:43.284179    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" event={"ID":"391b8927-4198-458a-92b4-e3528d751abe","Type":"ContainerStarted","Data":"084152f0bf97f17e4dd7e3c0b355bfa1a24f11baeb306c5abb7f7180d6932432"}
Jul 24 17:48:43.356401 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:43.356341    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" podStartSLOduration=1.889203863 podStartE2EDuration="5.356323179s" totalImagesPullingTime="3.467119316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:43.354401507 +0000 UTC m=+530.559093305" watchObservedRunningTime="2026-07-24 17:48:43.356323179 +0000 UTC m=+530.561014975"
Jul 24 17:48:43.474901 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:43.474811    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:44.479735 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:44.479705    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:45.300141 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:45.300108    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:45.301049 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:45.301027    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:48:54.443828 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.443797    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"]
Jul 24 17:48:54.450078 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.450056    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:48:54.452508 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.452486    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 17:48:54.460093 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.460068    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"]
Jul 24 17:48:54.545102 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.545064    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"]
Jul 24 17:48:54.569276 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.569250    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh978\" (UniqueName: \"kubernetes.io/projected/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-kube-api-access-lh978\") pod \"limitador-limitador-64c94fd64f-5sc9q\" (UID: \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:48:54.569418 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.569312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-config-file\") pod \"limitador-limitador-64c94fd64f-5sc9q\" (UID: \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:48:54.669984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.669954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh978\" (UniqueName: \"kubernetes.io/projected/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-kube-api-access-lh978\") pod \"limitador-limitador-64c94fd64f-5sc9q\" (UID: \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:48:54.670119 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.670013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-config-file\") pod \"limitador-limitador-64c94fd64f-5sc9q\" (UID: \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:48:54.670686 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.670666    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-config-file\") pod \"limitador-limitador-64c94fd64f-5sc9q\" (UID: \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:48:54.678465 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.678442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh978\" (UniqueName: \"kubernetes.io/projected/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-kube-api-access-lh978\") pod \"limitador-limitador-64c94fd64f-5sc9q\" (UID: \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:48:54.761167 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.761111    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:48:54.895618 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:54.895589    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"]
Jul 24 17:48:54.896943 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:48:54.896919    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2a1a17_bbfe_4ee3_bc42_4efc7a782f71.slice/crio-86b65f16728d93f6065e102b9f8a2e1db175ba0934cab13efb93f834f92d48a8 WatchSource:0}: Error finding container 86b65f16728d93f6065e102b9f8a2e1db175ba0934cab13efb93f834f92d48a8: Status 404 returned error can't find the container with id 86b65f16728d93f6065e102b9f8a2e1db175ba0934cab13efb93f834f92d48a8
Jul 24 17:48:55.243875 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.243846    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6l7w8"]
Jul 24 17:48:55.247194 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.247171    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6l7w8"
Jul 24 17:48:55.250521 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.250503    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cnq2w\""
Jul 24 17:48:55.256717 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.256681    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6l7w8"]
Jul 24 17:48:55.365356 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.365317    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q" event={"ID":"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71","Type":"ContainerStarted","Data":"86b65f16728d93f6065e102b9f8a2e1db175ba0934cab13efb93f834f92d48a8"}
Jul 24 17:48:55.376071 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.376043    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njg7q\" (UniqueName: \"kubernetes.io/projected/cca6b400-7dea-41d2-9015-4da2f5548a18-kube-api-access-njg7q\") pod \"authorino-58cd955675-6l7w8\" (UID: \"cca6b400-7dea-41d2-9015-4da2f5548a18\") " pod="kuadrant-system/authorino-58cd955675-6l7w8"
Jul 24 17:48:55.476808 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.476775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njg7q\" (UniqueName: \"kubernetes.io/projected/cca6b400-7dea-41d2-9015-4da2f5548a18-kube-api-access-njg7q\") pod \"authorino-58cd955675-6l7w8\" (UID: \"cca6b400-7dea-41d2-9015-4da2f5548a18\") " pod="kuadrant-system/authorino-58cd955675-6l7w8"
Jul 24 17:48:55.484683 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.484659    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njg7q\" (UniqueName: \"kubernetes.io/projected/cca6b400-7dea-41d2-9015-4da2f5548a18-kube-api-access-njg7q\") pod \"authorino-58cd955675-6l7w8\" (UID: \"cca6b400-7dea-41d2-9015-4da2f5548a18\") " pod="kuadrant-system/authorino-58cd955675-6l7w8"
Jul 24 17:48:55.554403 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.554369    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hgfj"]
Jul 24 17:48:55.557757 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.557732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6l7w8"
Jul 24 17:48:55.558597 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.558577    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7hgfj"
Jul 24 17:48:55.564310 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.564288    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hgfj"]
Jul 24 17:48:55.679317 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.679238    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwb8z\" (UniqueName: \"kubernetes.io/projected/c5a79c2b-597e-4ddb-8110-ca91c4238e5b-kube-api-access-kwb8z\") pod \"authorino-75d556bd77-7hgfj\" (UID: \"c5a79c2b-597e-4ddb-8110-ca91c4238e5b\") " pod="kuadrant-system/authorino-75d556bd77-7hgfj"
Jul 24 17:48:55.754486 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.754455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6l7w8"]
Jul 24 17:48:55.779974 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.779942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwb8z\" (UniqueName: \"kubernetes.io/projected/c5a79c2b-597e-4ddb-8110-ca91c4238e5b-kube-api-access-kwb8z\") pod \"authorino-75d556bd77-7hgfj\" (UID: \"c5a79c2b-597e-4ddb-8110-ca91c4238e5b\") " pod="kuadrant-system/authorino-75d556bd77-7hgfj"
Jul 24 17:48:55.790214 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.790130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwb8z\" (UniqueName: \"kubernetes.io/projected/c5a79c2b-597e-4ddb-8110-ca91c4238e5b-kube-api-access-kwb8z\") pod \"authorino-75d556bd77-7hgfj\" (UID: \"c5a79c2b-597e-4ddb-8110-ca91c4238e5b\") " pod="kuadrant-system/authorino-75d556bd77-7hgfj"
Jul 24 17:48:55.884517 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:55.884454    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7hgfj"
Jul 24 17:48:56.040244 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:56.040214    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hgfj"]
Jul 24 17:48:56.040986 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:48:56.040958    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5a79c2b_597e_4ddb_8110_ca91c4238e5b.slice/crio-ed22114f6865a3ca1b0b7ba5cd48884d66097267656799a60518702d05383c04 WatchSource:0}: Error finding container ed22114f6865a3ca1b0b7ba5cd48884d66097267656799a60518702d05383c04: Status 404 returned error can't find the container with id ed22114f6865a3ca1b0b7ba5cd48884d66097267656799a60518702d05383c04
Jul 24 17:48:56.372896 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:56.372842    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6l7w8" event={"ID":"cca6b400-7dea-41d2-9015-4da2f5548a18","Type":"ContainerStarted","Data":"9ec3a51a14be419a7943bf21215c518db5b17dc6e7587ed8bd5340279e7629ee"}
Jul 24 17:48:56.374154 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:56.374123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7hgfj" event={"ID":"c5a79c2b-597e-4ddb-8110-ca91c4238e5b","Type":"ContainerStarted","Data":"ed22114f6865a3ca1b0b7ba5cd48884d66097267656799a60518702d05383c04"}
Jul 24 17:48:58.395117 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:58.395075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q" event={"ID":"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71","Type":"ContainerStarted","Data":"70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9"}
Jul 24 17:48:58.395577 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:58.395312    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:48:58.417475 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:48:58.416690    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q" podStartSLOduration=1.4547373989999999 podStartE2EDuration="4.416671439s" totalImagesPullingTime="2.96193404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:58.412686081 +0000 UTC m=+545.617377880" watchObservedRunningTime="2026-07-24 17:48:58.416671439 +0000 UTC m=+545.621363238"
Jul 24 17:49:00.410741 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:00.410657    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6l7w8" event={"ID":"cca6b400-7dea-41d2-9015-4da2f5548a18","Type":"ContainerStarted","Data":"174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3"}
Jul 24 17:49:00.412328 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:00.412298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7hgfj" event={"ID":"c5a79c2b-597e-4ddb-8110-ca91c4238e5b","Type":"ContainerStarted","Data":"2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119"}
Jul 24 17:49:00.426740 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:00.426695    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6l7w8" podStartSLOduration=1.073556854 podStartE2EDuration="5.426681616s" totalImagesPullingTime="4.353124762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:00.425290086 +0000 UTC m=+547.629981886" watchObservedRunningTime="2026-07-24 17:49:00.426681616 +0000 UTC m=+547.631373412"
Jul 24 17:49:00.438408 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:00.438363    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7hgfj" podStartSLOduration=1.379024783 podStartE2EDuration="5.438350035s" totalImagesPullingTime="4.059325252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:00.438295403 +0000 UTC m=+547.642987212" watchObservedRunningTime="2026-07-24 17:49:00.438350035 +0000 UTC m=+547.643041831"
Jul 24 17:49:00.465670 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:00.465638    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6l7w8"]
Jul 24 17:49:02.423113 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:02.423073    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6l7w8" podUID="cca6b400-7dea-41d2-9015-4da2f5548a18" containerName="authorino" containerID="cri-o://174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3" gracePeriod=30
Jul 24 17:49:02.668138 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:02.668114    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6l7w8"
Jul 24 17:49:02.740562 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:02.740508    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njg7q\" (UniqueName: \"kubernetes.io/projected/cca6b400-7dea-41d2-9015-4da2f5548a18-kube-api-access-njg7q\") pod \"cca6b400-7dea-41d2-9015-4da2f5548a18\" (UID: \"cca6b400-7dea-41d2-9015-4da2f5548a18\") "
Jul 24 17:49:02.742402 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:02.742377    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca6b400-7dea-41d2-9015-4da2f5548a18-kube-api-access-njg7q" (OuterVolumeSpecName: "kube-api-access-njg7q") pod "cca6b400-7dea-41d2-9015-4da2f5548a18" (UID: "cca6b400-7dea-41d2-9015-4da2f5548a18"). InnerVolumeSpecName "kube-api-access-njg7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:02.841961 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:02.841940    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njg7q\" (UniqueName: \"kubernetes.io/projected/cca6b400-7dea-41d2-9015-4da2f5548a18-kube-api-access-njg7q\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:49:03.430780 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:03.430744    2562 generic.go:356] "Generic (PLEG): container finished" podID="cca6b400-7dea-41d2-9015-4da2f5548a18" containerID="174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3" exitCode=0
Jul 24 17:49:03.431145 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:03.430790    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6l7w8"
Jul 24 17:49:03.431145 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:03.430835    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6l7w8" event={"ID":"cca6b400-7dea-41d2-9015-4da2f5548a18","Type":"ContainerDied","Data":"174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3"}
Jul 24 17:49:03.431145 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:03.430898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6l7w8" event={"ID":"cca6b400-7dea-41d2-9015-4da2f5548a18","Type":"ContainerDied","Data":"9ec3a51a14be419a7943bf21215c518db5b17dc6e7587ed8bd5340279e7629ee"}
Jul 24 17:49:03.431145 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:03.430919    2562 scope.go:117] "RemoveContainer" containerID="174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3"
Jul 24 17:49:03.440555 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:03.440535    2562 scope.go:117] "RemoveContainer" containerID="174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3"
Jul 24 17:49:03.440857 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:49:03.440836    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3\": container with ID starting with 174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3 not found: ID does not exist" containerID="174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3"
Jul 24 17:49:03.440939 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:03.440869    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3"} err="failed to get container status \"174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3\": rpc error: code = NotFound desc = could not find container \"174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3\": container with ID starting with 174ca1a8243b92288c5925a9f6566f54c793709146e2b91bdb538eabdaf330b3 not found: ID does not exist"
Jul 24 17:49:03.449055 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:03.449031    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6l7w8"]
Jul 24 17:49:03.451513 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:03.451492    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6l7w8"]
Jul 24 17:49:05.407382 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:05.407343    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cca6b400-7dea-41d2-9015-4da2f5548a18" path="/var/lib/kubelet/pods/cca6b400-7dea-41d2-9015-4da2f5548a18/volumes"
Jul 24 17:49:09.407695 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:09.407660    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:49:09.755964 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:09.755865    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"]
Jul 24 17:49:09.756121 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:09.756046    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q" podUID="5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71" containerName="limitador" containerID="cri-o://70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9" gracePeriod=30
Jul 24 17:49:10.305300 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.305276    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:49:10.398326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.398247    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-config-file\") pod \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\" (UID: \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\") "
Jul 24 17:49:10.398326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.398316    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh978\" (UniqueName: \"kubernetes.io/projected/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-kube-api-access-lh978\") pod \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\" (UID: \"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71\") "
Jul 24 17:49:10.398583 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.398559    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-config-file" (OuterVolumeSpecName: "config-file") pod "5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71" (UID: "5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:49:10.400326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.400300    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-kube-api-access-lh978" (OuterVolumeSpecName: "kube-api-access-lh978") pod "5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71" (UID: "5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71"). InnerVolumeSpecName "kube-api-access-lh978". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:10.470071 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.470047    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71" containerID="70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9" exitCode=0
Jul 24 17:49:10.470404 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.470082    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"
Jul 24 17:49:10.470404 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.470127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q" event={"ID":"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71","Type":"ContainerDied","Data":"70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9"}
Jul 24 17:49:10.470404 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.470170    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5sc9q" event={"ID":"5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71","Type":"ContainerDied","Data":"86b65f16728d93f6065e102b9f8a2e1db175ba0934cab13efb93f834f92d48a8"}
Jul 24 17:49:10.470404 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.470192    2562 scope.go:117] "RemoveContainer" containerID="70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9"
Jul 24 17:49:10.479193 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.479177    2562 scope.go:117] "RemoveContainer" containerID="70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9"
Jul 24 17:49:10.479438 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:49:10.479418    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9\": container with ID starting with 70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9 not found: ID does not exist" containerID="70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9"
Jul 24 17:49:10.479487 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.479446    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9"} err="failed to get container status \"70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9\": rpc error: code = NotFound desc = could not find container \"70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9\": container with ID starting with 70fbbe0064334fd439e80ac1a3b2e663d3116a829fa3614fa2e1b22bc96b91f9 not found: ID does not exist"
Jul 24 17:49:10.491994 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.491971    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"]
Jul 24 17:49:10.499504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.499480    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh978\" (UniqueName: \"kubernetes.io/projected/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-kube-api-access-lh978\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:49:10.499504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.499502    2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71-config-file\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:49:10.500572 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:10.500553    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5sc9q"]
Jul 24 17:49:11.409605 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.409561    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71" path="/var/lib/kubelet/pods/5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71/volumes"
Jul 24 17:49:11.452830 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.452803    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"]
Jul 24 17:49:11.453213 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.453200    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cca6b400-7dea-41d2-9015-4da2f5548a18" containerName="authorino"
Jul 24 17:49:11.453260 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.453215    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cca6b400-7dea-41d2-9015-4da2f5548a18" containerName="authorino"
Jul 24 17:49:11.453260 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.453234    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71" containerName="limitador"
Jul 24 17:49:11.453260 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.453239    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71" containerName="limitador"
Jul 24 17:49:11.453361 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.453302    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c2a1a17-bbfe-4ee3-bc42-4efc7a782f71" containerName="limitador"
Jul 24 17:49:11.453361 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.453315    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cca6b400-7dea-41d2-9015-4da2f5548a18" containerName="authorino"
Jul 24 17:49:11.457006 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.456979    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:11.465399 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.465377    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-wjdgh\""
Jul 24 17:49:11.477807 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.477780    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 24 17:49:11.477807 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.477795    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:49:11.480466 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.480443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"]
Jul 24 17:49:11.481859 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.481843    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:49:11.608036 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.608008    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99xhs\" (UniqueName: \"kubernetes.io/projected/ae4ad542-6006-4ce2-9ae1-8f0b092ccc08-kube-api-access-99xhs\") pod \"postgres-69fb4594cf-sjlz2\" (UID: \"ae4ad542-6006-4ce2-9ae1-8f0b092ccc08\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:11.608182 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.608068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ae4ad542-6006-4ce2-9ae1-8f0b092ccc08-data\") pod \"postgres-69fb4594cf-sjlz2\" (UID: \"ae4ad542-6006-4ce2-9ae1-8f0b092ccc08\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:11.709308 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.709244    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ae4ad542-6006-4ce2-9ae1-8f0b092ccc08-data\") pod \"postgres-69fb4594cf-sjlz2\" (UID: \"ae4ad542-6006-4ce2-9ae1-8f0b092ccc08\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:11.709308 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.709297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99xhs\" (UniqueName: \"kubernetes.io/projected/ae4ad542-6006-4ce2-9ae1-8f0b092ccc08-kube-api-access-99xhs\") pod \"postgres-69fb4594cf-sjlz2\" (UID: \"ae4ad542-6006-4ce2-9ae1-8f0b092ccc08\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:11.709659 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.709634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ae4ad542-6006-4ce2-9ae1-8f0b092ccc08-data\") pod \"postgres-69fb4594cf-sjlz2\" (UID: \"ae4ad542-6006-4ce2-9ae1-8f0b092ccc08\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:11.718214 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.718186    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99xhs\" (UniqueName: \"kubernetes.io/projected/ae4ad542-6006-4ce2-9ae1-8f0b092ccc08-kube-api-access-99xhs\") pod \"postgres-69fb4594cf-sjlz2\" (UID: \"ae4ad542-6006-4ce2-9ae1-8f0b092ccc08\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:11.766852 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:11.766830    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:12.106464 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:12.106441    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"]
Jul 24 17:49:12.108264 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:49:12.108233    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae4ad542_6006_4ce2_9ae1_8f0b092ccc08.slice/crio-fdd9365d5daf1f34a7e30406a4d2d840cf7ec28ac303482d60247bbf9e6bf51f WatchSource:0}: Error finding container fdd9365d5daf1f34a7e30406a4d2d840cf7ec28ac303482d60247bbf9e6bf51f: Status 404 returned error can't find the container with id fdd9365d5daf1f34a7e30406a4d2d840cf7ec28ac303482d60247bbf9e6bf51f
Jul 24 17:49:12.484429 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:12.484359    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2" event={"ID":"ae4ad542-6006-4ce2-9ae1-8f0b092ccc08","Type":"ContainerStarted","Data":"fdd9365d5daf1f34a7e30406a4d2d840cf7ec28ac303482d60247bbf9e6bf51f"}
Jul 24 17:49:17.519393 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:17.519311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2" event={"ID":"ae4ad542-6006-4ce2-9ae1-8f0b092ccc08","Type":"ContainerStarted","Data":"357ede208dcf13ca236e7ecc762f53af369cb855ea7667699e9e8b355e2567ad"}
Jul 24 17:49:17.519393 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:17.519381    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:17.538912 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:17.538852    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2" podStartSLOduration=1.443991643 podStartE2EDuration="6.538838417s" totalImagesPullingTime="5.094846774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:17.537247744 +0000 UTC m=+564.741939540" watchObservedRunningTime="2026-07-24 17:49:17.538838417 +0000 UTC m=+564.743530214"
Jul 24 17:49:23.553011 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:23.552983    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjlz2"
Jul 24 17:49:24.159296 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.159261    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vpx5g"]
Jul 24 17:49:24.162937 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.162898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vpx5g"
Jul 24 17:49:24.170294 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.170159    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vpx5g"]
Jul 24 17:49:24.311777 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.311749    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqwld\" (UniqueName: \"kubernetes.io/projected/6cecf809-360d-4c2a-8904-f9e020af3d98-kube-api-access-sqwld\") pod \"authorino-5d4b67c446-vpx5g\" (UID: \"6cecf809-360d-4c2a-8904-f9e020af3d98\") " pod="kuadrant-system/authorino-5d4b67c446-vpx5g"
Jul 24 17:49:24.352628 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.352606    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vpx5g"]
Jul 24 17:49:24.352800 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:49:24.352784    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sqwld], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-vpx5g" podUID="6cecf809-360d-4c2a-8904-f9e020af3d98"
Jul 24 17:49:24.413117 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.413054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqwld\" (UniqueName: \"kubernetes.io/projected/6cecf809-360d-4c2a-8904-f9e020af3d98-kube-api-access-sqwld\") pod \"authorino-5d4b67c446-vpx5g\" (UID: \"6cecf809-360d-4c2a-8904-f9e020af3d98\") " pod="kuadrant-system/authorino-5d4b67c446-vpx5g"
Jul 24 17:49:24.422776 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.422756    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqwld\" (UniqueName: \"kubernetes.io/projected/6cecf809-360d-4c2a-8904-f9e020af3d98-kube-api-access-sqwld\") pod \"authorino-5d4b67c446-vpx5g\" (UID: \"6cecf809-360d-4c2a-8904-f9e020af3d98\") " pod="kuadrant-system/authorino-5d4b67c446-vpx5g"
Jul 24 17:49:24.556730 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.556706    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vpx5g"
Jul 24 17:49:24.563452 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.563431    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vpx5g"
Jul 24 17:49:24.716172 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.716116    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqwld\" (UniqueName: \"kubernetes.io/projected/6cecf809-360d-4c2a-8904-f9e020af3d98-kube-api-access-sqwld\") pod \"6cecf809-360d-4c2a-8904-f9e020af3d98\" (UID: \"6cecf809-360d-4c2a-8904-f9e020af3d98\") "
Jul 24 17:49:24.718151 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.718124    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cecf809-360d-4c2a-8904-f9e020af3d98-kube-api-access-sqwld" (OuterVolumeSpecName: "kube-api-access-sqwld") pod "6cecf809-360d-4c2a-8904-f9e020af3d98" (UID: "6cecf809-360d-4c2a-8904-f9e020af3d98"). InnerVolumeSpecName "kube-api-access-sqwld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:24.817098 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:24.817075    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqwld\" (UniqueName: \"kubernetes.io/projected/6cecf809-360d-4c2a-8904-f9e020af3d98-kube-api-access-sqwld\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:49:25.561732 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:25.561705    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vpx5g"
Jul 24 17:49:25.591291 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:25.591265    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vpx5g"]
Jul 24 17:49:25.593832 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:25.593810    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vpx5g"]
Jul 24 17:49:27.409229 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:27.409186    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cecf809-360d-4c2a-8904-f9e020af3d98" path="/var/lib/kubelet/pods/6cecf809-360d-4c2a-8904-f9e020af3d98/volumes"
Jul 24 17:49:27.923686 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:27.923645    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hgfj"]
Jul 24 17:49:27.923893 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:27.923833    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7hgfj" podUID="c5a79c2b-597e-4ddb-8110-ca91c4238e5b" containerName="authorino" containerID="cri-o://2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119" gracePeriod=30
Jul 24 17:49:28.188494 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.188431    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7hgfj"
Jul 24 17:49:28.260767 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.260738    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwb8z\" (UniqueName: \"kubernetes.io/projected/c5a79c2b-597e-4ddb-8110-ca91c4238e5b-kube-api-access-kwb8z\") pod \"c5a79c2b-597e-4ddb-8110-ca91c4238e5b\" (UID: \"c5a79c2b-597e-4ddb-8110-ca91c4238e5b\") "
Jul 24 17:49:28.262719 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.262696    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5a79c2b-597e-4ddb-8110-ca91c4238e5b-kube-api-access-kwb8z" (OuterVolumeSpecName: "kube-api-access-kwb8z") pod "c5a79c2b-597e-4ddb-8110-ca91c4238e5b" (UID: "c5a79c2b-597e-4ddb-8110-ca91c4238e5b"). InnerVolumeSpecName "kube-api-access-kwb8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:28.361484 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.361454    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwb8z\" (UniqueName: \"kubernetes.io/projected/c5a79c2b-597e-4ddb-8110-ca91c4238e5b-kube-api-access-kwb8z\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:49:28.580797 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.580767    2562 generic.go:356] "Generic (PLEG): container finished" podID="c5a79c2b-597e-4ddb-8110-ca91c4238e5b" containerID="2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119" exitCode=0
Jul 24 17:49:28.581185 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.580804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7hgfj" event={"ID":"c5a79c2b-597e-4ddb-8110-ca91c4238e5b","Type":"ContainerDied","Data":"2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119"}
Jul 24 17:49:28.581185 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.580819    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7hgfj"
Jul 24 17:49:28.581185 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.580848    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7hgfj" event={"ID":"c5a79c2b-597e-4ddb-8110-ca91c4238e5b","Type":"ContainerDied","Data":"ed22114f6865a3ca1b0b7ba5cd48884d66097267656799a60518702d05383c04"}
Jul 24 17:49:28.581185 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.580865    2562 scope.go:117] "RemoveContainer" containerID="2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119"
Jul 24 17:49:28.589822 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.589806    2562 scope.go:117] "RemoveContainer" containerID="2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119"
Jul 24 17:49:28.590104 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:49:28.590083    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119\": container with ID starting with 2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119 not found: ID does not exist" containerID="2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119"
Jul 24 17:49:28.590183 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.590110    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119"} err="failed to get container status \"2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119\": rpc error: code = NotFound desc = could not find container \"2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119\": container with ID starting with 2a9507fd05715d87e0d70452b812806d40aca9fa709d626d8273f1f3ff216119 not found: ID does not exist"
Jul 24 17:49:28.610282 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.610260    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hgfj"]
Jul 24 17:49:28.620748 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:28.620730    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hgfj"]
Jul 24 17:49:29.408122 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:29.408088    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5a79c2b-597e-4ddb-8110-ca91c4238e5b" path="/var/lib/kubelet/pods/c5a79c2b-597e-4ddb-8110-ca91c4238e5b/volumes"
Jul 24 17:49:32.949501 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.949471    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-546465d569-k8fq5"]
Jul 24 17:49:32.949956 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.949903    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5a79c2b-597e-4ddb-8110-ca91c4238e5b" containerName="authorino"
Jul 24 17:49:32.949956 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.949915    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5a79c2b-597e-4ddb-8110-ca91c4238e5b" containerName="authorino"
Jul 24 17:49:32.950034 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.950002    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5a79c2b-597e-4ddb-8110-ca91c4238e5b" containerName="authorino"
Jul 24 17:49:32.956662 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.956640    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:32.959428 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.959400    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 17:49:32.960397 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.960376    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z2qkg\""
Jul 24 17:49:32.960522 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.960425    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 17:49:32.963332 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.963314    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-546465d569-k8fq5"]
Jul 24 17:49:32.991901 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.991859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed6ef70-8b5b-421f-9956-90d953aaad44-webhook-cert\") pod \"maas-controller-546465d569-k8fq5\" (UID: \"3ed6ef70-8b5b-421f-9956-90d953aaad44\") " pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:32.992017 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:32.991956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8lxs\" (UniqueName: \"kubernetes.io/projected/3ed6ef70-8b5b-421f-9956-90d953aaad44-kube-api-access-r8lxs\") pod \"maas-controller-546465d569-k8fq5\" (UID: \"3ed6ef70-8b5b-421f-9956-90d953aaad44\") " pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:33.092931 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.092901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed6ef70-8b5b-421f-9956-90d953aaad44-webhook-cert\") pod \"maas-controller-546465d569-k8fq5\" (UID: \"3ed6ef70-8b5b-421f-9956-90d953aaad44\") " pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:33.093092 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.092998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lxs\" (UniqueName: \"kubernetes.io/projected/3ed6ef70-8b5b-421f-9956-90d953aaad44-kube-api-access-r8lxs\") pod \"maas-controller-546465d569-k8fq5\" (UID: \"3ed6ef70-8b5b-421f-9956-90d953aaad44\") " pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:33.095323 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.095303    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed6ef70-8b5b-421f-9956-90d953aaad44-webhook-cert\") pod \"maas-controller-546465d569-k8fq5\" (UID: \"3ed6ef70-8b5b-421f-9956-90d953aaad44\") " pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:33.103762 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.103740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8lxs\" (UniqueName: \"kubernetes.io/projected/3ed6ef70-8b5b-421f-9956-90d953aaad44-kube-api-access-r8lxs\") pod \"maas-controller-546465d569-k8fq5\" (UID: \"3ed6ef70-8b5b-421f-9956-90d953aaad44\") " pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:33.128464 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.128438    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84d669b67d-ttfwv"]
Jul 24 17:49:33.130984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.130968    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:33.145407 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.145383    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d669b67d-ttfwv"]
Jul 24 17:49:33.193324 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.193280    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7690f5b6-52b8-484c-bad5-728d36919715-webhook-cert\") pod \"maas-controller-84d669b67d-ttfwv\" (UID: \"7690f5b6-52b8-484c-bad5-728d36919715\") " pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:33.193324 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.193331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtflt\" (UniqueName: \"kubernetes.io/projected/7690f5b6-52b8-484c-bad5-728d36919715-kube-api-access-rtflt\") pod \"maas-controller-84d669b67d-ttfwv\" (UID: \"7690f5b6-52b8-484c-bad5-728d36919715\") " pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:33.267688 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.267616    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:33.293793 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.293766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7690f5b6-52b8-484c-bad5-728d36919715-webhook-cert\") pod \"maas-controller-84d669b67d-ttfwv\" (UID: \"7690f5b6-52b8-484c-bad5-728d36919715\") " pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:33.293921 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.293814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtflt\" (UniqueName: \"kubernetes.io/projected/7690f5b6-52b8-484c-bad5-728d36919715-kube-api-access-rtflt\") pod \"maas-controller-84d669b67d-ttfwv\" (UID: \"7690f5b6-52b8-484c-bad5-728d36919715\") " pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:33.296078 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.296054    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7690f5b6-52b8-484c-bad5-728d36919715-webhook-cert\") pod \"maas-controller-84d669b67d-ttfwv\" (UID: \"7690f5b6-52b8-484c-bad5-728d36919715\") " pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:33.305303 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.305276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtflt\" (UniqueName: \"kubernetes.io/projected/7690f5b6-52b8-484c-bad5-728d36919715-kube-api-access-rtflt\") pod \"maas-controller-84d669b67d-ttfwv\" (UID: \"7690f5b6-52b8-484c-bad5-728d36919715\") " pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:33.399625 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.399603    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-546465d569-k8fq5"]
Jul 24 17:49:33.401141 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:49:33.401109    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed6ef70_8b5b_421f_9956_90d953aaad44.slice/crio-433c2b59332f91a3034fd5a336a397e5b4a20e8cd924cf993045198bf4fcfd56 WatchSource:0}: Error finding container 433c2b59332f91a3034fd5a336a397e5b4a20e8cd924cf993045198bf4fcfd56: Status 404 returned error can't find the container with id 433c2b59332f91a3034fd5a336a397e5b4a20e8cd924cf993045198bf4fcfd56
Jul 24 17:49:33.440807 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.440774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:33.579526 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.579495    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d669b67d-ttfwv"]
Jul 24 17:49:33.581006 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:49:33.580985    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7690f5b6_52b8_484c_bad5_728d36919715.slice/crio-b39ecc4ef5c561dafc34af78eec4fbe5d069ffba18f6e28a4b6ee6f345eb9edf WatchSource:0}: Error finding container b39ecc4ef5c561dafc34af78eec4fbe5d069ffba18f6e28a4b6ee6f345eb9edf: Status 404 returned error can't find the container with id b39ecc4ef5c561dafc34af78eec4fbe5d069ffba18f6e28a4b6ee6f345eb9edf
Jul 24 17:49:33.612005 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.611970    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-546465d569-k8fq5" event={"ID":"3ed6ef70-8b5b-421f-9956-90d953aaad44","Type":"ContainerStarted","Data":"433c2b59332f91a3034fd5a336a397e5b4a20e8cd924cf993045198bf4fcfd56"}
Jul 24 17:49:33.613164 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:33.613141    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d669b67d-ttfwv" event={"ID":"7690f5b6-52b8-484c-bad5-728d36919715","Type":"ContainerStarted","Data":"b39ecc4ef5c561dafc34af78eec4fbe5d069ffba18f6e28a4b6ee6f345eb9edf"}
Jul 24 17:49:36.637797 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:36.637752    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-546465d569-k8fq5" event={"ID":"3ed6ef70-8b5b-421f-9956-90d953aaad44","Type":"ContainerStarted","Data":"0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb"}
Jul 24 17:49:36.638273 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:36.637835    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:36.639382 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:36.639360    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d669b67d-ttfwv" event={"ID":"7690f5b6-52b8-484c-bad5-728d36919715","Type":"ContainerStarted","Data":"3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039"}
Jul 24 17:49:36.639512 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:36.639487    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:36.660772 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:36.660722    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-546465d569-k8fq5" podStartSLOduration=2.192997453 podStartE2EDuration="4.660706561s" totalImagesPullingTime="2.467709108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:36.658122212 +0000 UTC m=+583.862814010" watchObservedRunningTime="2026-07-24 17:49:36.660706561 +0000 UTC m=+583.865398364"
Jul 24 17:49:36.679406 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:36.679359    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84d669b67d-ttfwv" podStartSLOduration=1.384481232 podStartE2EDuration="3.679344867s" totalImagesPullingTime="2.294863635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:36.678399016 +0000 UTC m=+583.883090813" watchObservedRunningTime="2026-07-24 17:49:36.679344867 +0000 UTC m=+583.884036664"
Jul 24 17:49:47.646760 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:47.646728    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:47.647230 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:47.646830    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:49:47.718229 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:47.718173    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-546465d569-k8fq5"]
Jul 24 17:49:47.718420 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:47.718399    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-546465d569-k8fq5" podUID="3ed6ef70-8b5b-421f-9956-90d953aaad44" containerName="manager" containerID="cri-o://0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb" gracePeriod=30
Jul 24 17:49:48.013685 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.013659    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:48.117597 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.117570    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed6ef70-8b5b-421f-9956-90d953aaad44-webhook-cert\") pod \"3ed6ef70-8b5b-421f-9956-90d953aaad44\" (UID: \"3ed6ef70-8b5b-421f-9956-90d953aaad44\") "
Jul 24 17:49:48.117760 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.117621    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8lxs\" (UniqueName: \"kubernetes.io/projected/3ed6ef70-8b5b-421f-9956-90d953aaad44-kube-api-access-r8lxs\") pod \"3ed6ef70-8b5b-421f-9956-90d953aaad44\" (UID: \"3ed6ef70-8b5b-421f-9956-90d953aaad44\") "
Jul 24 17:49:48.119756 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.119719    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ed6ef70-8b5b-421f-9956-90d953aaad44-kube-api-access-r8lxs" (OuterVolumeSpecName: "kube-api-access-r8lxs") pod "3ed6ef70-8b5b-421f-9956-90d953aaad44" (UID: "3ed6ef70-8b5b-421f-9956-90d953aaad44"). InnerVolumeSpecName "kube-api-access-r8lxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:48.119756 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.119730    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ed6ef70-8b5b-421f-9956-90d953aaad44-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3ed6ef70-8b5b-421f-9956-90d953aaad44" (UID: "3ed6ef70-8b5b-421f-9956-90d953aaad44"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:49:48.219091 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.219027    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed6ef70-8b5b-421f-9956-90d953aaad44-webhook-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:49:48.219091 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.219055    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8lxs\" (UniqueName: \"kubernetes.io/projected/3ed6ef70-8b5b-421f-9956-90d953aaad44-kube-api-access-r8lxs\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:49:48.717261 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.717227    2562 generic.go:356] "Generic (PLEG): container finished" podID="3ed6ef70-8b5b-421f-9956-90d953aaad44" containerID="0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb" exitCode=0
Jul 24 17:49:48.717639 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.717263    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-546465d569-k8fq5"
Jul 24 17:49:48.717639 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.717309    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-546465d569-k8fq5" event={"ID":"3ed6ef70-8b5b-421f-9956-90d953aaad44","Type":"ContainerDied","Data":"0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb"}
Jul 24 17:49:48.717639 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.717358    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-546465d569-k8fq5" event={"ID":"3ed6ef70-8b5b-421f-9956-90d953aaad44","Type":"ContainerDied","Data":"433c2b59332f91a3034fd5a336a397e5b4a20e8cd924cf993045198bf4fcfd56"}
Jul 24 17:49:48.717639 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.717387    2562 scope.go:117] "RemoveContainer" containerID="0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb"
Jul 24 17:49:48.727087 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.726817    2562 scope.go:117] "RemoveContainer" containerID="0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb"
Jul 24 17:49:48.727141 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:49:48.727093    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb\": container with ID starting with 0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb not found: ID does not exist" containerID="0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb"
Jul 24 17:49:48.727141 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.727119    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb"} err="failed to get container status \"0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb\": rpc error: code = NotFound desc = could not find container \"0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb\": container with ID starting with 0dbc7bb4cd1e7d6e017866ac56dd410df67a1cca314f3b7bf544573b0d6525eb not found: ID does not exist"
Jul 24 17:49:48.740663 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.740643    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-546465d569-k8fq5"]
Jul 24 17:49:48.744587 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:48.744566    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-546465d569-k8fq5"]
Jul 24 17:49:49.408010 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:49.407970    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ed6ef70-8b5b-421f-9956-90d953aaad44" path="/var/lib/kubelet/pods/3ed6ef70-8b5b-421f-9956-90d953aaad44/volumes"
Jul 24 17:49:53.245445 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:53.245414    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:49:53.246098 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:53.246078    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:49:53.253965 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:53.253942    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:49:53.254281 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:49:53.254267    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:50:07.972522 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:07.972488    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"]
Jul 24 17:50:07.979953 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:07.972931    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed6ef70-8b5b-421f-9956-90d953aaad44" containerName="manager"
Jul 24 17:50:07.979953 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:07.972944    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed6ef70-8b5b-421f-9956-90d953aaad44" containerName="manager"
Jul 24 17:50:07.979953 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:07.973007    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed6ef70-8b5b-421f-9956-90d953aaad44" containerName="manager"
Jul 24 17:50:07.981037 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:07.981020    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:07.984903 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:07.984849    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 24 17:50:07.985277 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:07.985253    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ppc8g\""
Jul 24 17:50:07.987707 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:07.987688    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"]
Jul 24 17:50:08.079445 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.079413    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e72bfd-0350-4439-ac35-6cd5bd27f6ff-openshift-service-ca\") pod \"maas-api-5c8f99fbb4-kr6kk\" (UID: \"72e72bfd-0350-4439-ac35-6cd5bd27f6ff\") " pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.079445 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.079446    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4f4n\" (UniqueName: \"kubernetes.io/projected/72e72bfd-0350-4439-ac35-6cd5bd27f6ff-kube-api-access-x4f4n\") pod \"maas-api-5c8f99fbb4-kr6kk\" (UID: \"72e72bfd-0350-4439-ac35-6cd5bd27f6ff\") " pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.079668 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.079474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e72bfd-0350-4439-ac35-6cd5bd27f6ff-maas-api-tls\") pod \"maas-api-5c8f99fbb4-kr6kk\" (UID: \"72e72bfd-0350-4439-ac35-6cd5bd27f6ff\") " pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.180122 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.180083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e72bfd-0350-4439-ac35-6cd5bd27f6ff-openshift-service-ca\") pod \"maas-api-5c8f99fbb4-kr6kk\" (UID: \"72e72bfd-0350-4439-ac35-6cd5bd27f6ff\") " pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.180122 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.180126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4f4n\" (UniqueName: \"kubernetes.io/projected/72e72bfd-0350-4439-ac35-6cd5bd27f6ff-kube-api-access-x4f4n\") pod \"maas-api-5c8f99fbb4-kr6kk\" (UID: \"72e72bfd-0350-4439-ac35-6cd5bd27f6ff\") " pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.180372 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.180166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e72bfd-0350-4439-ac35-6cd5bd27f6ff-maas-api-tls\") pod \"maas-api-5c8f99fbb4-kr6kk\" (UID: \"72e72bfd-0350-4439-ac35-6cd5bd27f6ff\") " pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.180898 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.180859    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e72bfd-0350-4439-ac35-6cd5bd27f6ff-openshift-service-ca\") pod \"maas-api-5c8f99fbb4-kr6kk\" (UID: \"72e72bfd-0350-4439-ac35-6cd5bd27f6ff\") " pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.182753 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.182731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e72bfd-0350-4439-ac35-6cd5bd27f6ff-maas-api-tls\") pod \"maas-api-5c8f99fbb4-kr6kk\" (UID: \"72e72bfd-0350-4439-ac35-6cd5bd27f6ff\") " pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.194484 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.194457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4f4n\" (UniqueName: \"kubernetes.io/projected/72e72bfd-0350-4439-ac35-6cd5bd27f6ff-kube-api-access-x4f4n\") pod \"maas-api-5c8f99fbb4-kr6kk\" (UID: \"72e72bfd-0350-4439-ac35-6cd5bd27f6ff\") " pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.292387 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.292360    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:08.441240 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.441212    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"]
Jul 24 17:50:08.441994 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:50:08.441965    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e72bfd_0350_4439_ac35_6cd5bd27f6ff.slice/crio-56f60c058fc76a1a3adb02ef91c1d63077a93ce0685138144f58d78bad0e6e3f WatchSource:0}: Error finding container 56f60c058fc76a1a3adb02ef91c1d63077a93ce0685138144f58d78bad0e6e3f: Status 404 returned error can't find the container with id 56f60c058fc76a1a3adb02ef91c1d63077a93ce0685138144f58d78bad0e6e3f
Jul 24 17:50:08.443526 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.443507    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:50:08.554606 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.554520    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"]
Jul 24 17:50:08.559728 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.559712    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:08.562739 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.562718    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jfk7v\""
Jul 24 17:50:08.562899 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.562857    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 17:50:08.576678 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.576650    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"]
Jul 24 17:50:08.684446 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.684411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2299da18-3500-4fa7-bb76-dd8ad6086623-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-wll8r\" (UID: \"2299da18-3500-4fa7-bb76-dd8ad6086623\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:08.684630 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.684463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkpkn\" (UniqueName: \"kubernetes.io/projected/2299da18-3500-4fa7-bb76-dd8ad6086623-kube-api-access-jkpkn\") pod \"payload-pre-processing-687dbb6cf6-wll8r\" (UID: \"2299da18-3500-4fa7-bb76-dd8ad6086623\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:08.693099 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.693069    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-w5rtf"]
Jul 24 17:50:08.697715 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.697696    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:08.709601 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.709573    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-w5rtf"]
Jul 24 17:50:08.785523 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.785487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2299da18-3500-4fa7-bb76-dd8ad6086623-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-wll8r\" (UID: \"2299da18-3500-4fa7-bb76-dd8ad6086623\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:08.785692 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.785526    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f82zx\" (UniqueName: \"kubernetes.io/projected/1e1e7f00-128d-419c-8297-673e02231338-kube-api-access-f82zx\") pod \"payload-processing-5cc6cd444b-w5rtf\" (UID: \"1e1e7f00-128d-419c-8297-673e02231338\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:08.785692 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.785551    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e1e7f00-128d-419c-8297-673e02231338-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-w5rtf\" (UID: \"1e1e7f00-128d-419c-8297-673e02231338\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:08.785692 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.785605    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkpkn\" (UniqueName: \"kubernetes.io/projected/2299da18-3500-4fa7-bb76-dd8ad6086623-kube-api-access-jkpkn\") pod \"payload-pre-processing-687dbb6cf6-wll8r\" (UID: \"2299da18-3500-4fa7-bb76-dd8ad6086623\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:08.786248 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.786230    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2299da18-3500-4fa7-bb76-dd8ad6086623-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-wll8r\" (UID: \"2299da18-3500-4fa7-bb76-dd8ad6086623\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:08.796025 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.796005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkpkn\" (UniqueName: \"kubernetes.io/projected/2299da18-3500-4fa7-bb76-dd8ad6086623-kube-api-access-jkpkn\") pod \"payload-pre-processing-687dbb6cf6-wll8r\" (UID: \"2299da18-3500-4fa7-bb76-dd8ad6086623\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:08.828168 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.828105    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk" event={"ID":"72e72bfd-0350-4439-ac35-6cd5bd27f6ff","Type":"ContainerStarted","Data":"56f60c058fc76a1a3adb02ef91c1d63077a93ce0685138144f58d78bad0e6e3f"}
Jul 24 17:50:08.869973 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.869948    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:08.886465 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.886437    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f82zx\" (UniqueName: \"kubernetes.io/projected/1e1e7f00-128d-419c-8297-673e02231338-kube-api-access-f82zx\") pod \"payload-processing-5cc6cd444b-w5rtf\" (UID: \"1e1e7f00-128d-419c-8297-673e02231338\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:08.886581 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.886475    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e1e7f00-128d-419c-8297-673e02231338-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-w5rtf\" (UID: \"1e1e7f00-128d-419c-8297-673e02231338\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:08.887135 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.887117    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e1e7f00-128d-419c-8297-673e02231338-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-w5rtf\" (UID: \"1e1e7f00-128d-419c-8297-673e02231338\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:08.894445 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:08.894426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f82zx\" (UniqueName: \"kubernetes.io/projected/1e1e7f00-128d-419c-8297-673e02231338-kube-api-access-f82zx\") pod \"payload-processing-5cc6cd444b-w5rtf\" (UID: \"1e1e7f00-128d-419c-8297-673e02231338\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:09.011487 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:09.011445    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:09.221101 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:09.221070    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"]
Jul 24 17:50:09.221789 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:50:09.221760    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2299da18_3500_4fa7_bb76_dd8ad6086623.slice/crio-071bd43e1ed192b9f530d9b38c1828cd30868a8acb750ec35f33004960d49562 WatchSource:0}: Error finding container 071bd43e1ed192b9f530d9b38c1828cd30868a8acb750ec35f33004960d49562: Status 404 returned error can't find the container with id 071bd43e1ed192b9f530d9b38c1828cd30868a8acb750ec35f33004960d49562
Jul 24 17:50:09.399081 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:09.399054    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-w5rtf"]
Jul 24 17:50:09.399872 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:50:09.399779    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e1e7f00_128d_419c_8297_673e02231338.slice/crio-b9e529045591aa2c884bfd0c935f436e036a2d32e4c311e6190d466d7c048045 WatchSource:0}: Error finding container b9e529045591aa2c884bfd0c935f436e036a2d32e4c311e6190d466d7c048045: Status 404 returned error can't find the container with id b9e529045591aa2c884bfd0c935f436e036a2d32e4c311e6190d466d7c048045
Jul 24 17:50:09.839435 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:09.839398    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r" event={"ID":"2299da18-3500-4fa7-bb76-dd8ad6086623","Type":"ContainerStarted","Data":"071bd43e1ed192b9f530d9b38c1828cd30868a8acb750ec35f33004960d49562"}
Jul 24 17:50:09.841179 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:09.841114    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf" event={"ID":"1e1e7f00-128d-419c-8297-673e02231338","Type":"ContainerStarted","Data":"b9e529045591aa2c884bfd0c935f436e036a2d32e4c311e6190d466d7c048045"}
Jul 24 17:50:10.871003 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:10.869972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk" event={"ID":"72e72bfd-0350-4439-ac35-6cd5bd27f6ff","Type":"ContainerStarted","Data":"2ae50132e1bda427332405c6139b66192b35083eccd9ca263546638670c28f9f"}
Jul 24 17:50:10.871003 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:10.870774    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:10.889961 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:10.889914    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk" podStartSLOduration=1.851245181 podStartE2EDuration="3.889893686s" totalImagesPullingTime="2.038648505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:10.887618799 +0000 UTC m=+618.092310597" watchObservedRunningTime="2026-07-24 17:50:10.889893686 +0000 UTC m=+618.094585475"
Jul 24 17:50:11.879682 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:11.879648    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r" event={"ID":"2299da18-3500-4fa7-bb76-dd8ad6086623","Type":"ContainerStarted","Data":"0d48e9ff1538b68b28995aaf991cf71bae76f6250ead30083b58d9a0eaa36843"}
Jul 24 17:50:11.880166 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:11.879848    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:11.881348 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:11.881321    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf" event={"ID":"1e1e7f00-128d-419c-8297-673e02231338","Type":"ContainerStarted","Data":"733d8b817b22a825484c3432079e888b5377a241afaeb0b1be458ce03bce4265"}
Jul 24 17:50:11.881492 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:11.881475    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:11.899798 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:11.899758    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r" podStartSLOduration=1.622027413 podStartE2EDuration="3.899744358s" totalImagesPullingTime="2.277716945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:11.895558721 +0000 UTC m=+619.100250518" watchObservedRunningTime="2026-07-24 17:50:11.899744358 +0000 UTC m=+619.104436154"
Jul 24 17:50:11.914786 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:11.914744    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf" podStartSLOduration=1.688783827 podStartE2EDuration="3.914731441s" totalImagesPullingTime="2.225947614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:11.913211842 +0000 UTC m=+619.117903636" watchObservedRunningTime="2026-07-24 17:50:11.914731441 +0000 UTC m=+619.119423238"
Jul 24 17:50:15.772487 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:15.772045    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qbgk5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:53928->10.132.0.54:15021: read: connection reset by peer" start-of-body=
Jul 24 17:50:15.772487 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:15.772115    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" podUID="391b8927-4198-458a-92b4-e3528d751abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:53928->10.132.0.54:15021: read: connection reset by peer"
Jul 24 17:50:15.908175 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:15.908146    2562 generic.go:356] "Generic (PLEG): container finished" podID="391b8927-4198-458a-92b4-e3528d751abe" containerID="084152f0bf97f17e4dd7e3c0b355bfa1a24f11baeb306c5abb7f7180d6932432" exitCode=0
Jul 24 17:50:15.908287 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:15.908190    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" event={"ID":"391b8927-4198-458a-92b4-e3528d751abe","Type":"ContainerDied","Data":"084152f0bf97f17e4dd7e3c0b355bfa1a24f11baeb306c5abb7f7180d6932432"}
Jul 24 17:50:15.908556 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:15.908540    2562 scope.go:117] "RemoveContainer" containerID="084152f0bf97f17e4dd7e3c0b355bfa1a24f11baeb306c5abb7f7180d6932432"
Jul 24 17:50:15.909449 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:15.909414    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:50:15.909514 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:15.909493    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:50:15.909550 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:15.909524    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:50:16.915411 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:16.915372    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" event={"ID":"391b8927-4198-458a-92b4-e3528d751abe","Type":"ContainerStarted","Data":"b8309538de56335e0b8128fb789aa6e5e6510fcb3f64abbeaf0c0615fea9bfe0"}
Jul 24 17:50:17.475336 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:17.475298    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:50:17.475537 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:17.475514    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qbgk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 24 17:50:17.475591 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:17.475573    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" podUID="391b8927-4198-458a-92b4-e3528d751abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 24 17:50:17.893620 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:17.893592    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c8f99fbb4-kr6kk"
Jul 24 17:50:18.475189 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:18.475155    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qbgk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 24 17:50:18.475578 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:18.475219    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" podUID="391b8927-4198-458a-92b4-e3528d751abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 24 17:50:19.475428 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:19.475392    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qbgk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 24 17:50:19.475800 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:19.475471    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" podUID="391b8927-4198-458a-92b4-e3528d751abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 24 17:50:20.475776 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:20.475733    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qbgk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 24 17:50:20.476177 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:20.475798    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5" podUID="391b8927-4198-458a-92b4-e3528d751abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 24 17:50:21.478277 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:21.478242    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:50:21.478652 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:21.478634    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:50:21.479338 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:21.479319    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qbgk5"
Jul 24 17:50:22.890115 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:22.890083    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-w5rtf"
Jul 24 17:50:22.890506 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:22.890211    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-wll8r"
Jul 24 17:50:23.992232 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:23.992205    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"]
Jul 24 17:50:23.996145 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:23.996124    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:23.998709 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:23.998688    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 24 17:50:23.998836 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:23.998782    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 17:50:23.999622 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:23.999602    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 17:50:23.999729 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:23.999608    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k52rs\""
Jul 24 17:50:24.006396 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.006372    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"]
Jul 24 17:50:24.126821 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.126791    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.126988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.126831    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.126988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.126853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae657de7-9977-4f12-8346-5e5df2c79355-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.126988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.126895    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xcxg\" (UniqueName: \"kubernetes.io/projected/ae657de7-9977-4f12-8346-5e5df2c79355-kube-api-access-4xcxg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.126988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.126916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.126988 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.126956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.227986 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.227957    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xcxg\" (UniqueName: \"kubernetes.io/projected/ae657de7-9977-4f12-8346-5e5df2c79355-kube-api-access-4xcxg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.228134 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.227993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.228134 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.228041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.228134 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.228103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.228301 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.228142    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.228301 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.228174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae657de7-9977-4f12-8346-5e5df2c79355-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.228538 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.228514    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.228625 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.228547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.228625 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.228565    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.230213 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.230192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae657de7-9977-4f12-8346-5e5df2c79355-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.230481 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.230462    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae657de7-9977-4f12-8346-5e5df2c79355-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.236125 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.236105    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xcxg\" (UniqueName: \"kubernetes.io/projected/ae657de7-9977-4f12-8346-5e5df2c79355-kube-api-access-4xcxg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6\" (UID: \"ae657de7-9977-4f12-8346-5e5df2c79355\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.307241 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.307217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:24.442381 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.442354    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"]
Jul 24 17:50:24.443523 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:50:24.443496    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae657de7_9977_4f12_8346_5e5df2c79355.slice/crio-eb0617b963347a09422fa148c71806b56359452d298bdab5a1389c33afe29a8a WatchSource:0}: Error finding container eb0617b963347a09422fa148c71806b56359452d298bdab5a1389c33afe29a8a: Status 404 returned error can't find the container with id eb0617b963347a09422fa148c71806b56359452d298bdab5a1389c33afe29a8a
Jul 24 17:50:24.968605 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:24.968567    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6" event={"ID":"ae657de7-9977-4f12-8346-5e5df2c79355","Type":"ContainerStarted","Data":"eb0617b963347a09422fa148c71806b56359452d298bdab5a1389c33afe29a8a"}
Jul 24 17:50:25.981922 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:25.981822    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6" event={"ID":"ae657de7-9977-4f12-8346-5e5df2c79355","Type":"ContainerStarted","Data":"11745c18fa23106c74ad4cef7f5e5f0437165d2c1ed0e9e7982b3e28a0731409"}
Jul 24 17:50:26.002676 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:26.002629    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6" podStartSLOduration=1.796373425 podStartE2EDuration="3.002616568s" totalImagesPullingTime="1.206243143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:26.000341424 +0000 UTC m=+633.205033247" watchObservedRunningTime="2026-07-24 17:50:26.002616568 +0000 UTC m=+633.207308365"
Jul 24 17:50:34.307357 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:34.307320    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:34.307756 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:34.307395    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:34.319693 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:34.319674    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:35.041869 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:35.041835    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6"
Jul 24 17:50:58.297680 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.297647    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"]
Jul 24 17:50:58.302042 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.302026    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.304778 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.304758    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 24 17:50:58.310623 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.310602    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"]
Jul 24 17:50:58.405543 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.405508    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.405683 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.405575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59fxn\" (UniqueName: \"kubernetes.io/projected/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-kube-api-access-59fxn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.405683 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.405603    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.405683 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.405628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.405789 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.405720    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.405789 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.405748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.507034 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.507001    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.507197 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.507074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fxn\" (UniqueName: \"kubernetes.io/projected/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-kube-api-access-59fxn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.507197 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.507100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.507197 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.507126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.507197 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.507167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.507445 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.507356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.507445 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.507409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.507529 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.507451    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.507665 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.507647    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.509439 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.509420    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.509596 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.509578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.516336 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.516313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59fxn\" (UniqueName: \"kubernetes.io/projected/f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd-kube-api-access-59fxn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd\" (UID: \"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.613931 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.613856    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:50:58.957834 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:58.957807    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"]
Jul 24 17:50:58.959013 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:50:58.958986    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ccc6e4_341d_4e7c_ae8e_2bf6afa957cd.slice/crio-cfcfbe9caf05440c660778e79a17d08dd44e39e835aed08cb72faa34938789b8 WatchSource:0}: Error finding container cfcfbe9caf05440c660778e79a17d08dd44e39e835aed08cb72faa34938789b8: Status 404 returned error can't find the container with id cfcfbe9caf05440c660778e79a17d08dd44e39e835aed08cb72faa34938789b8
Jul 24 17:50:59.165672 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:50:59.165638    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd" event={"ID":"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd","Type":"ContainerStarted","Data":"cfcfbe9caf05440c660778e79a17d08dd44e39e835aed08cb72faa34938789b8"}
Jul 24 17:51:00.172576 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:51:00.172544    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd" event={"ID":"f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd","Type":"ContainerStarted","Data":"c8372528e2c380126e98f0c9d93954618beb3825556138b4e65849cf9e9a735f"}
Jul 24 17:51:00.205100 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:51:00.205049    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd" podStartSLOduration=2.000115311 podStartE2EDuration="2.205034453s" totalImagesPullingTime="204.919142ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:51:00.202669122 +0000 UTC m=+667.407360920" watchObservedRunningTime="2026-07-24 17:51:00.205034453 +0000 UTC m=+667.409726250"
Jul 24 17:51:08.614554 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:51:08.614522    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:51:08.615039 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:51:08.614588    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:51:08.627132 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:51:08.627106    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:51:09.235368 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:51:09.235336    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd"
Jul 24 17:53:40.113987 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.113957    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65bfbcb984-gjmwg"]
Jul 24 17:53:40.117381 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.117365    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65bfbcb984-gjmwg"
Jul 24 17:53:40.121113 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.121092    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 17:53:40.121220 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.121092    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cnq2w\""
Jul 24 17:53:40.134052 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.134030    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65bfbcb984-gjmwg"]
Jul 24 17:53:40.150504 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.150482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80f83c3e-d8e1-465a-8d9e-a471a04c72e9-tls-cert\") pod \"authorino-65bfbcb984-gjmwg\" (UID: \"80f83c3e-d8e1-465a-8d9e-a471a04c72e9\") " pod="kuadrant-system/authorino-65bfbcb984-gjmwg"
Jul 24 17:53:40.150625 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.150511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqskt\" (UniqueName: \"kubernetes.io/projected/80f83c3e-d8e1-465a-8d9e-a471a04c72e9-kube-api-access-hqskt\") pod \"authorino-65bfbcb984-gjmwg\" (UID: \"80f83c3e-d8e1-465a-8d9e-a471a04c72e9\") " pod="kuadrant-system/authorino-65bfbcb984-gjmwg"
Jul 24 17:53:40.251766 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.251743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqskt\" (UniqueName: \"kubernetes.io/projected/80f83c3e-d8e1-465a-8d9e-a471a04c72e9-kube-api-access-hqskt\") pod \"authorino-65bfbcb984-gjmwg\" (UID: \"80f83c3e-d8e1-465a-8d9e-a471a04c72e9\") " pod="kuadrant-system/authorino-65bfbcb984-gjmwg"
Jul 24 17:53:40.251863 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.251843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80f83c3e-d8e1-465a-8d9e-a471a04c72e9-tls-cert\") pod \"authorino-65bfbcb984-gjmwg\" (UID: \"80f83c3e-d8e1-465a-8d9e-a471a04c72e9\") " pod="kuadrant-system/authorino-65bfbcb984-gjmwg"
Jul 24 17:53:40.254211 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.254185    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80f83c3e-d8e1-465a-8d9e-a471a04c72e9-tls-cert\") pod \"authorino-65bfbcb984-gjmwg\" (UID: \"80f83c3e-d8e1-465a-8d9e-a471a04c72e9\") " pod="kuadrant-system/authorino-65bfbcb984-gjmwg"
Jul 24 17:53:40.260543 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.260523    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqskt\" (UniqueName: \"kubernetes.io/projected/80f83c3e-d8e1-465a-8d9e-a471a04c72e9-kube-api-access-hqskt\") pod \"authorino-65bfbcb984-gjmwg\" (UID: \"80f83c3e-d8e1-465a-8d9e-a471a04c72e9\") " pod="kuadrant-system/authorino-65bfbcb984-gjmwg"
Jul 24 17:53:40.426392 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.426338    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65bfbcb984-gjmwg"
Jul 24 17:53:40.566629 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:40.566604    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65bfbcb984-gjmwg"]
Jul 24 17:53:40.568111 ip-10-0-136-89 kubenswrapper[2562]: W0724 17:53:40.568083    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80f83c3e_d8e1_465a_8d9e_a471a04c72e9.slice/crio-87d2d287ecd12ef15b6f5a1022df14d95a99d8fa9b00f529edda3533d1fb4418 WatchSource:0}: Error finding container 87d2d287ecd12ef15b6f5a1022df14d95a99d8fa9b00f529edda3533d1fb4418: Status 404 returned error can't find the container with id 87d2d287ecd12ef15b6f5a1022df14d95a99d8fa9b00f529edda3533d1fb4418
Jul 24 17:53:41.103092 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:41.103053    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65bfbcb984-gjmwg" event={"ID":"80f83c3e-d8e1-465a-8d9e-a471a04c72e9","Type":"ContainerStarted","Data":"4ab043960aa9fbe423551ebddf4142714600981970d84f2e34c0f2c8d0f13342"}
Jul 24 17:53:41.103260 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:41.103098    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65bfbcb984-gjmwg" event={"ID":"80f83c3e-d8e1-465a-8d9e-a471a04c72e9","Type":"ContainerStarted","Data":"87d2d287ecd12ef15b6f5a1022df14d95a99d8fa9b00f529edda3533d1fb4418"}
Jul 24 17:53:41.119531 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:53:41.119483    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65bfbcb984-gjmwg" podStartSLOduration=0.754091081 podStartE2EDuration="1.119468871s" totalImagesPullingTime="365.37779ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:53:41.118061592 +0000 UTC m=+828.322753388" watchObservedRunningTime="2026-07-24 17:53:41.119468871 +0000 UTC m=+828.324160700"
Jul 24 17:54:30.744343 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:30.744311    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d669b67d-ttfwv"]
Jul 24 17:54:30.744853 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:30.744590    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84d669b67d-ttfwv" podUID="7690f5b6-52b8-484c-bad5-728d36919715" containerName="manager" containerID="cri-o://3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039" gracePeriod=30
Jul 24 17:54:31.004102 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.004046    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:54:31.093536 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.093501    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtflt\" (UniqueName: \"kubernetes.io/projected/7690f5b6-52b8-484c-bad5-728d36919715-kube-api-access-rtflt\") pod \"7690f5b6-52b8-484c-bad5-728d36919715\" (UID: \"7690f5b6-52b8-484c-bad5-728d36919715\") "
Jul 24 17:54:31.093665 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.093542    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7690f5b6-52b8-484c-bad5-728d36919715-webhook-cert\") pod \"7690f5b6-52b8-484c-bad5-728d36919715\" (UID: \"7690f5b6-52b8-484c-bad5-728d36919715\") "
Jul 24 17:54:31.095584 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.095551    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7690f5b6-52b8-484c-bad5-728d36919715-kube-api-access-rtflt" (OuterVolumeSpecName: "kube-api-access-rtflt") pod "7690f5b6-52b8-484c-bad5-728d36919715" (UID: "7690f5b6-52b8-484c-bad5-728d36919715"). InnerVolumeSpecName "kube-api-access-rtflt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:54:31.095584 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.095571    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7690f5b6-52b8-484c-bad5-728d36919715-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7690f5b6-52b8-484c-bad5-728d36919715" (UID: "7690f5b6-52b8-484c-bad5-728d36919715"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:54:31.195142 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.195118    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtflt\" (UniqueName: \"kubernetes.io/projected/7690f5b6-52b8-484c-bad5-728d36919715-kube-api-access-rtflt\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:54:31.195142 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.195141    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7690f5b6-52b8-484c-bad5-728d36919715-webhook-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 17:54:31.397788 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.397764    2562 generic.go:356] "Generic (PLEG): container finished" podID="7690f5b6-52b8-484c-bad5-728d36919715" containerID="3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039" exitCode=0
Jul 24 17:54:31.397923 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.397803    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d669b67d-ttfwv"
Jul 24 17:54:31.397923 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.397849    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d669b67d-ttfwv" event={"ID":"7690f5b6-52b8-484c-bad5-728d36919715","Type":"ContainerDied","Data":"3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039"}
Jul 24 17:54:31.398010 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.397923    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d669b67d-ttfwv" event={"ID":"7690f5b6-52b8-484c-bad5-728d36919715","Type":"ContainerDied","Data":"b39ecc4ef5c561dafc34af78eec4fbe5d069ffba18f6e28a4b6ee6f345eb9edf"}
Jul 24 17:54:31.398010 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.397952    2562 scope.go:117] "RemoveContainer" containerID="3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039"
Jul 24 17:54:31.408017 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.408002    2562 scope.go:117] "RemoveContainer" containerID="3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039"
Jul 24 17:54:31.408293 ip-10-0-136-89 kubenswrapper[2562]: E0724 17:54:31.408275    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039\": container with ID starting with 3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039 not found: ID does not exist" containerID="3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039"
Jul 24 17:54:31.408343 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.408300    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039"} err="failed to get container status \"3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039\": rpc error: code = NotFound desc = could not find container \"3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039\": container with ID starting with 3fa85b3feb7324d8af9aed561a9178e7a886a5813a02993574b4e6c188426039 not found: ID does not exist"
Jul 24 17:54:31.420408 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.420383    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d669b67d-ttfwv"]
Jul 24 17:54:31.424029 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:31.424009    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84d669b67d-ttfwv"]
Jul 24 17:54:33.410833 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:33.410793    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7690f5b6-52b8-484c-bad5-728d36919715" path="/var/lib/kubelet/pods/7690f5b6-52b8-484c-bad5-728d36919715/volumes"
Jul 24 17:54:53.288313 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:53.288287    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:54:53.289910 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:53.289868    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:54:53.297351 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:53.297326    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:54:53.298907 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:54:53.298870    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:59:53.327280 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:59:53.327247    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:59:53.334514 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:59:53.334488    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 17:59:53.335984 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:59:53.335958    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 17:59:53.343326 ip-10-0-136-89 kubenswrapper[2562]: I0724 17:59:53.343308    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:04:53.364296 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:04:53.364265    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:04:53.372690 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:04:53.372668    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:04:53.373715 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:04:53.373689    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:04:53.381179 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:04:53.381162    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:08:52.990994 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:52.990960    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"]
Jul 24 18:08:52.993429 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:52.991357    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7690f5b6-52b8-484c-bad5-728d36919715" containerName="manager"
Jul 24 18:08:52.993429 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:52.991368    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7690f5b6-52b8-484c-bad5-728d36919715" containerName="manager"
Jul 24 18:08:52.993429 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:52.991449    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7690f5b6-52b8-484c-bad5-728d36919715" containerName="manager"
Jul 24 18:08:52.994413 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:52.994397    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:08:52.996722 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:52.996699    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3e8e6244\""
Jul 24 18:08:53.005169 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.005143    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"]
Jul 24 18:08:53.035201 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.035175    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"]
Jul 24 18:08:53.038562 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.038547    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:08:53.041183 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.041169    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3e8e6244-dockercfg-svxdt\""
Jul 24 18:08:53.051469 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.051448    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"]
Jul 24 18:08:53.116529 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.116503    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffv2v\" (UniqueName: \"kubernetes.io/projected/192a9206-da45-465b-8492-65680fe8333e-kube-api-access-ffv2v\") pod \"payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb\" (UID: \"192a9206-da45-465b-8492-65680fe8333e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:08:53.116686 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.116666    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a9206-da45-465b-8492-65680fe8333e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb\" (UID: \"192a9206-da45-465b-8492-65680fe8333e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:08:53.116757 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.116701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79082377-6c14-451c-bb36-2af55be01bf3-plugins-config-volume\") pod \"payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv\" (UID: \"79082377-6c14-451c-bb36-2af55be01bf3\") " pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:08:53.116757 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.116734    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2lwc\" (UniqueName: \"kubernetes.io/projected/79082377-6c14-451c-bb36-2af55be01bf3-kube-api-access-b2lwc\") pod \"payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv\" (UID: \"79082377-6c14-451c-bb36-2af55be01bf3\") " pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:08:53.218103 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.218066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a9206-da45-465b-8492-65680fe8333e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb\" (UID: \"192a9206-da45-465b-8492-65680fe8333e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:08:53.218103 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.218109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79082377-6c14-451c-bb36-2af55be01bf3-plugins-config-volume\") pod \"payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv\" (UID: \"79082377-6c14-451c-bb36-2af55be01bf3\") " pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:08:53.218275 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.218127    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2lwc\" (UniqueName: \"kubernetes.io/projected/79082377-6c14-451c-bb36-2af55be01bf3-kube-api-access-b2lwc\") pod \"payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv\" (UID: \"79082377-6c14-451c-bb36-2af55be01bf3\") " pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:08:53.218275 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.218246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffv2v\" (UniqueName: \"kubernetes.io/projected/192a9206-da45-465b-8492-65680fe8333e-kube-api-access-ffv2v\") pod \"payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb\" (UID: \"192a9206-da45-465b-8492-65680fe8333e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:08:53.220733 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.220710    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3e8e6244\""
Jul 24 18:08:53.227195 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.227174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffv2v\" (UniqueName: \"kubernetes.io/projected/192a9206-da45-465b-8492-65680fe8333e-kube-api-access-ffv2v\") pod \"payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb\" (UID: \"192a9206-da45-465b-8492-65680fe8333e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:08:53.227353 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.227330    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2lwc\" (UniqueName: \"kubernetes.io/projected/79082377-6c14-451c-bb36-2af55be01bf3-kube-api-access-b2lwc\") pod \"payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv\" (UID: \"79082377-6c14-451c-bb36-2af55be01bf3\") " pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:08:53.229678 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.229656    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79082377-6c14-451c-bb36-2af55be01bf3-plugins-config-volume\") pod \"payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv\" (UID: \"79082377-6c14-451c-bb36-2af55be01bf3\") " pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:08:53.229745 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.229684    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a9206-da45-465b-8492-65680fe8333e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb\" (UID: \"192a9206-da45-465b-8492-65680fe8333e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:08:53.306204 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.306176    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:08:53.350816 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.350784    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3e8e6244-dockercfg-svxdt\""
Jul 24 18:08:53.359258 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.359226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:08:53.654673 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.654645    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"]
Jul 24 18:08:53.655954 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:08:53.655930    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192a9206_da45_465b_8492_65680fe8333e.slice/crio-f507eef13198eb23e5beacdd45fef6375f5f290f9cc8c1e2dd9850522dc62704 WatchSource:0}: Error finding container f507eef13198eb23e5beacdd45fef6375f5f290f9cc8c1e2dd9850522dc62704: Status 404 returned error can't find the container with id f507eef13198eb23e5beacdd45fef6375f5f290f9cc8c1e2dd9850522dc62704
Jul 24 18:08:53.717704 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:53.717681    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"]
Jul 24 18:08:53.718990 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:08:53.718966    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79082377_6c14_451c_bb36_2af55be01bf3.slice/crio-fb47f9f67929f5f0f7e892eba38fa699ac2394e7a9488dfdf479489efe0ed7c3 WatchSource:0}: Error finding container fb47f9f67929f5f0f7e892eba38fa699ac2394e7a9488dfdf479489efe0ed7c3: Status 404 returned error can't find the container with id fb47f9f67929f5f0f7e892eba38fa699ac2394e7a9488dfdf479489efe0ed7c3
Jul 24 18:08:54.531186 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:54.531149    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv" event={"ID":"79082377-6c14-451c-bb36-2af55be01bf3","Type":"ContainerStarted","Data":"de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476"}
Jul 24 18:08:54.531606 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:54.531202    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv" event={"ID":"79082377-6c14-451c-bb36-2af55be01bf3","Type":"ContainerStarted","Data":"fb47f9f67929f5f0f7e892eba38fa699ac2394e7a9488dfdf479489efe0ed7c3"}
Jul 24 18:08:54.531606 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:54.531342    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:08:54.532773 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:54.532750    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb" event={"ID":"192a9206-da45-465b-8492-65680fe8333e","Type":"ContainerStarted","Data":"f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a"}
Jul 24 18:08:54.532896 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:54.532791    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb" event={"ID":"192a9206-da45-465b-8492-65680fe8333e","Type":"ContainerStarted","Data":"f507eef13198eb23e5beacdd45fef6375f5f290f9cc8c1e2dd9850522dc62704"}
Jul 24 18:08:54.532896 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:54.532837    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:08:54.550212 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:54.550170    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv" podStartSLOduration=1.550157556 podStartE2EDuration="1.550157556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:08:54.547325527 +0000 UTC m=+1741.752017325" watchObservedRunningTime="2026-07-24 18:08:54.550157556 +0000 UTC m=+1741.754849353"
Jul 24 18:08:54.565164 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:08:54.565127    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb" podStartSLOduration=2.565116039 podStartE2EDuration="2.565116039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:08:54.563784369 +0000 UTC m=+1741.768476167" watchObservedRunningTime="2026-07-24 18:08:54.565116039 +0000 UTC m=+1741.769807901"
Jul 24 18:09:05.540377 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:05.540346    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:09:05.540766 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:05.540417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:09:28.247258 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.247216    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"]
Jul 24 18:09:28.247860 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.247655    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv" podUID="79082377-6c14-451c-bb36-2af55be01bf3" containerName="payload-processing" containerID="cri-o://de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476" gracePeriod=30
Jul 24 18:09:28.255907 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.255044    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"]
Jul 24 18:09:28.255907 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.255243    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb" podUID="192a9206-da45-465b-8492-65680fe8333e" containerName="payload-pre-processing" containerID="cri-o://f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a" gracePeriod=30
Jul 24 18:09:28.538215 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.538190    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:09:28.541336 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.541314    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:09:28.594094 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.594071    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79082377-6c14-451c-bb36-2af55be01bf3-plugins-config-volume\") pod \"79082377-6c14-451c-bb36-2af55be01bf3\" (UID: \"79082377-6c14-451c-bb36-2af55be01bf3\") "
Jul 24 18:09:28.594234 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.594110    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2lwc\" (UniqueName: \"kubernetes.io/projected/79082377-6c14-451c-bb36-2af55be01bf3-kube-api-access-b2lwc\") pod \"79082377-6c14-451c-bb36-2af55be01bf3\" (UID: \"79082377-6c14-451c-bb36-2af55be01bf3\") "
Jul 24 18:09:28.594234 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.594127    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a9206-da45-465b-8492-65680fe8333e-plugins-config-volume\") pod \"192a9206-da45-465b-8492-65680fe8333e\" (UID: \"192a9206-da45-465b-8492-65680fe8333e\") "
Jul 24 18:09:28.594234 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.594214    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffv2v\" (UniqueName: \"kubernetes.io/projected/192a9206-da45-465b-8492-65680fe8333e-kube-api-access-ffv2v\") pod \"192a9206-da45-465b-8492-65680fe8333e\" (UID: \"192a9206-da45-465b-8492-65680fe8333e\") "
Jul 24 18:09:28.594486 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.594462    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79082377-6c14-451c-bb36-2af55be01bf3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79082377-6c14-451c-bb36-2af55be01bf3" (UID: "79082377-6c14-451c-bb36-2af55be01bf3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:09:28.594585 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.594556    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/192a9206-da45-465b-8492-65680fe8333e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "192a9206-da45-465b-8492-65680fe8333e" (UID: "192a9206-da45-465b-8492-65680fe8333e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:09:28.596387 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.596347    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79082377-6c14-451c-bb36-2af55be01bf3-kube-api-access-b2lwc" (OuterVolumeSpecName: "kube-api-access-b2lwc") pod "79082377-6c14-451c-bb36-2af55be01bf3" (UID: "79082377-6c14-451c-bb36-2af55be01bf3"). InnerVolumeSpecName "kube-api-access-b2lwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:09:28.596497 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.596383    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192a9206-da45-465b-8492-65680fe8333e-kube-api-access-ffv2v" (OuterVolumeSpecName: "kube-api-access-ffv2v") pod "192a9206-da45-465b-8492-65680fe8333e" (UID: "192a9206-da45-465b-8492-65680fe8333e"). InnerVolumeSpecName "kube-api-access-ffv2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:09:28.694940 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.694916    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffv2v\" (UniqueName: \"kubernetes.io/projected/192a9206-da45-465b-8492-65680fe8333e-kube-api-access-ffv2v\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:09:28.694940 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.694938    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79082377-6c14-451c-bb36-2af55be01bf3-plugins-config-volume\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:09:28.695088 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.694954    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2lwc\" (UniqueName: \"kubernetes.io/projected/79082377-6c14-451c-bb36-2af55be01bf3-kube-api-access-b2lwc\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:09:28.695088 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.694966    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a9206-da45-465b-8492-65680fe8333e-plugins-config-volume\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:09:28.741326 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.741299    2562 generic.go:356] "Generic (PLEG): container finished" podID="79082377-6c14-451c-bb36-2af55be01bf3" containerID="de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476" exitCode=0
Jul 24 18:09:28.741445 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.741343    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"
Jul 24 18:09:28.741445 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.741385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv" event={"ID":"79082377-6c14-451c-bb36-2af55be01bf3","Type":"ContainerDied","Data":"de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476"}
Jul 24 18:09:28.741445 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.741438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv" event={"ID":"79082377-6c14-451c-bb36-2af55be01bf3","Type":"ContainerDied","Data":"fb47f9f67929f5f0f7e892eba38fa699ac2394e7a9488dfdf479489efe0ed7c3"}
Jul 24 18:09:28.741592 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.741474    2562 scope.go:117] "RemoveContainer" containerID="de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476"
Jul 24 18:09:28.743079 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.743059    2562 generic.go:356] "Generic (PLEG): container finished" podID="192a9206-da45-465b-8492-65680fe8333e" containerID="f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a" exitCode=0
Jul 24 18:09:28.743186 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.743081    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb" event={"ID":"192a9206-da45-465b-8492-65680fe8333e","Type":"ContainerDied","Data":"f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a"}
Jul 24 18:09:28.743186 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.743103    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"
Jul 24 18:09:28.743186 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.743141    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb" event={"ID":"192a9206-da45-465b-8492-65680fe8333e","Type":"ContainerDied","Data":"f507eef13198eb23e5beacdd45fef6375f5f290f9cc8c1e2dd9850522dc62704"}
Jul 24 18:09:28.753217 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.753140    2562 scope.go:117] "RemoveContainer" containerID="de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476"
Jul 24 18:09:28.753477 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:09:28.753433    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476\": container with ID starting with de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476 not found: ID does not exist" containerID="de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476"
Jul 24 18:09:28.753554 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.753471    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476"} err="failed to get container status \"de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476\": rpc error: code = NotFound desc = could not find container \"de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476\": container with ID starting with de92db3fad554c7db8f06d189571cd54ffd2bb56ea6c58ea1b25a9644aae1476 not found: ID does not exist"
Jul 24 18:09:28.753554 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.753495    2562 scope.go:117] "RemoveContainer" containerID="f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a"
Jul 24 18:09:28.762152 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.762137    2562 scope.go:117] "RemoveContainer" containerID="f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a"
Jul 24 18:09:28.762395 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:09:28.762377    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a\": container with ID starting with f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a not found: ID does not exist" containerID="f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a"
Jul 24 18:09:28.762462 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.762401    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a"} err="failed to get container status \"f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a\": rpc error: code = NotFound desc = could not find container \"f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a\": container with ID starting with f1e8d0f40c34624d7e59349e05efecd20d81c6a914d825f19fea03f9ebb4285a not found: ID does not exist"
Jul 24 18:09:28.767787 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.767767    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"]
Jul 24 18:09:28.771592 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.771572    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3e8e6244-59cbddf9c4-fsmpv"]
Jul 24 18:09:28.781541 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.781519    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"]
Jul 24 18:09:28.786029 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:28.786011    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3e8e6244-6ddbdc796-f6kqb"]
Jul 24 18:09:29.414846 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:29.414811    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="192a9206-da45-465b-8492-65680fe8333e" path="/var/lib/kubelet/pods/192a9206-da45-465b-8492-65680fe8333e/volumes"
Jul 24 18:09:29.415358 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:29.415336    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79082377-6c14-451c-bb36-2af55be01bf3" path="/var/lib/kubelet/pods/79082377-6c14-451c-bb36-2af55be01bf3/volumes"
Jul 24 18:09:37.124287 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.124256    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"]
Jul 24 18:09:37.124702 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.124688    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192a9206-da45-465b-8492-65680fe8333e" containerName="payload-pre-processing"
Jul 24 18:09:37.124746 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.124704    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="192a9206-da45-465b-8492-65680fe8333e" containerName="payload-pre-processing"
Jul 24 18:09:37.124746 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.124733    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79082377-6c14-451c-bb36-2af55be01bf3" containerName="payload-processing"
Jul 24 18:09:37.124746 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.124739    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79082377-6c14-451c-bb36-2af55be01bf3" containerName="payload-processing"
Jul 24 18:09:37.124842 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.124817    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79082377-6c14-451c-bb36-2af55be01bf3" containerName="payload-processing"
Jul 24 18:09:37.124842 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.124827    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="192a9206-da45-465b-8492-65680fe8333e" containerName="payload-pre-processing"
Jul 24 18:09:37.129693 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.129673    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:37.132996 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.132976    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-8be87499-dockercfg-gwdx5\""
Jul 24 18:09:37.133442 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.133426    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-8be87499\""
Jul 24 18:09:37.140741 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.140718    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"]
Jul 24 18:09:37.158795 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.158770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4143506e-282e-4943-8fd2-f8a6f66c4c7f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8be87499-59b7657497-gvz56\" (UID: \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\") " pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:37.158917 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.158823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db6wc\" (UniqueName: \"kubernetes.io/projected/4143506e-282e-4943-8fd2-f8a6f66c4c7f-kube-api-access-db6wc\") pod \"payload-processing-e2e-migrate-8be87499-59b7657497-gvz56\" (UID: \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\") " pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:37.259749 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.259720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db6wc\" (UniqueName: \"kubernetes.io/projected/4143506e-282e-4943-8fd2-f8a6f66c4c7f-kube-api-access-db6wc\") pod \"payload-processing-e2e-migrate-8be87499-59b7657497-gvz56\" (UID: \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\") " pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:37.259910 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.259797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4143506e-282e-4943-8fd2-f8a6f66c4c7f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8be87499-59b7657497-gvz56\" (UID: \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\") " pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:37.260462 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.260443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4143506e-282e-4943-8fd2-f8a6f66c4c7f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8be87499-59b7657497-gvz56\" (UID: \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\") " pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:37.272379 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.272353    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db6wc\" (UniqueName: \"kubernetes.io/projected/4143506e-282e-4943-8fd2-f8a6f66c4c7f-kube-api-access-db6wc\") pod \"payload-processing-e2e-migrate-8be87499-59b7657497-gvz56\" (UID: \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\") " pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:37.441262 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.441200    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:37.785797 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:09:37.785760    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4143506e_282e_4943_8fd2_f8a6f66c4c7f.slice/crio-df98f8008f168f23fc88c126a502f52f67237040277249e4163c71a5af895c4a WatchSource:0}: Error finding container df98f8008f168f23fc88c126a502f52f67237040277249e4163c71a5af895c4a: Status 404 returned error can't find the container with id df98f8008f168f23fc88c126a502f52f67237040277249e4163c71a5af895c4a
Jul 24 18:09:37.787487 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.787417    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"]
Jul 24 18:09:37.802662 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:37.802629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56" event={"ID":"4143506e-282e-4943-8fd2-f8a6f66c4c7f","Type":"ContainerStarted","Data":"df98f8008f168f23fc88c126a502f52f67237040277249e4163c71a5af895c4a"}
Jul 24 18:09:38.810284 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:38.810247    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56" event={"ID":"4143506e-282e-4943-8fd2-f8a6f66c4c7f","Type":"ContainerStarted","Data":"f2bb740389c75e3022fe4003423c593352b02f82f836ab46ac309f2bb4fbf851"}
Jul 24 18:09:38.810728 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:38.810313    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:38.829794 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:38.829752    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56" podStartSLOduration=1.8297376010000002 podStartE2EDuration="1.829737601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:09:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:09:38.827186352 +0000 UTC m=+1786.031878148" watchObservedRunningTime="2026-07-24 18:09:38.829737601 +0000 UTC m=+1786.034429398"
Jul 24 18:09:49.818773 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:49.818738    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:09:53.402169 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:53.402138    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:09:53.419176 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:53.419153    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:09:53.421143 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:53.421120    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:09:53.429115 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:09:53.429095    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:10:11.879296 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:11.879262    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"]
Jul 24 18:10:11.879811 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:11.879483    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56" podUID="4143506e-282e-4943-8fd2-f8a6f66c4c7f" containerName="payload-processing" containerID="cri-o://f2bb740389c75e3022fe4003423c593352b02f82f836ab46ac309f2bb4fbf851" gracePeriod=30
Jul 24 18:10:12.013629 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:12.013599    2562 generic.go:356] "Generic (PLEG): container finished" podID="4143506e-282e-4943-8fd2-f8a6f66c4c7f" containerID="f2bb740389c75e3022fe4003423c593352b02f82f836ab46ac309f2bb4fbf851" exitCode=0
Jul 24 18:10:12.013769 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:12.013640    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56" event={"ID":"4143506e-282e-4943-8fd2-f8a6f66c4c7f","Type":"ContainerDied","Data":"f2bb740389c75e3022fe4003423c593352b02f82f836ab46ac309f2bb4fbf851"}
Jul 24 18:10:12.142378 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:12.142322    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:10:12.258415 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:12.258387    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db6wc\" (UniqueName: \"kubernetes.io/projected/4143506e-282e-4943-8fd2-f8a6f66c4c7f-kube-api-access-db6wc\") pod \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\" (UID: \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\") "
Jul 24 18:10:12.258599 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:12.258452    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4143506e-282e-4943-8fd2-f8a6f66c4c7f-plugins-config-volume\") pod \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\" (UID: \"4143506e-282e-4943-8fd2-f8a6f66c4c7f\") "
Jul 24 18:10:12.258858 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:12.258834    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4143506e-282e-4943-8fd2-f8a6f66c4c7f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4143506e-282e-4943-8fd2-f8a6f66c4c7f" (UID: "4143506e-282e-4943-8fd2-f8a6f66c4c7f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:10:12.260458 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:12.260438    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4143506e-282e-4943-8fd2-f8a6f66c4c7f-kube-api-access-db6wc" (OuterVolumeSpecName: "kube-api-access-db6wc") pod "4143506e-282e-4943-8fd2-f8a6f66c4c7f" (UID: "4143506e-282e-4943-8fd2-f8a6f66c4c7f"). InnerVolumeSpecName "kube-api-access-db6wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:10:12.359845 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:12.359813    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4143506e-282e-4943-8fd2-f8a6f66c4c7f-plugins-config-volume\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:10:12.359845 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:12.359838    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db6wc\" (UniqueName: \"kubernetes.io/projected/4143506e-282e-4943-8fd2-f8a6f66c4c7f-kube-api-access-db6wc\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:10:13.021743 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:13.021709    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56" event={"ID":"4143506e-282e-4943-8fd2-f8a6f66c4c7f","Type":"ContainerDied","Data":"df98f8008f168f23fc88c126a502f52f67237040277249e4163c71a5af895c4a"}
Jul 24 18:10:13.021743 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:13.021742    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"
Jul 24 18:10:13.022263 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:13.021769    2562 scope.go:117] "RemoveContainer" containerID="f2bb740389c75e3022fe4003423c593352b02f82f836ab46ac309f2bb4fbf851"
Jul 24 18:10:13.045798 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:13.045773    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"]
Jul 24 18:10:13.050110 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:13.050087    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8be87499-59b7657497-gvz56"]
Jul 24 18:10:13.408765 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:10:13.408721    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4143506e-282e-4943-8fd2-f8a6f66c4c7f" path="/var/lib/kubelet/pods/4143506e-282e-4943-8fd2-f8a6f66c4c7f/volumes"
Jul 24 18:14:53.452398 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:14:53.452369    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:14:53.460481 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:14:53.460461    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:14:53.463620 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:14:53.463601    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:14:53.471610 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:14:53.471593    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:15:00.134845 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.134813    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62"]
Jul 24 18:15:00.135364 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.135337    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4143506e-282e-4943-8fd2-f8a6f66c4c7f" containerName="payload-processing"
Jul 24 18:15:00.135364 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.135357    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4143506e-282e-4943-8fd2-f8a6f66c4c7f" containerName="payload-processing"
Jul 24 18:15:00.135506 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.135437    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4143506e-282e-4943-8fd2-f8a6f66c4c7f" containerName="payload-processing"
Jul 24 18:15:00.138369 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.138353    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn"]
Jul 24 18:15:00.138477 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.138458    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62"
Jul 24 18:15:00.141338 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.141320    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:15:00.141598 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.141579    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn"
Jul 24 18:15:00.149441 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.146903    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62"]
Jul 24 18:15:00.151050 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.150847    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn"]
Jul 24 18:15:00.155270 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.155250    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62"
Jul 24 18:15:00.162979 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.162959    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn"
Jul 24 18:15:00.512030 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:15:00.512001    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538b26ac_2494_4c0d_a62d_6e04c3a56bce.slice/crio-acbb9c5d3ad720d95e73ad1a7648a26a4c754d2bf325d71de4d9b14742ae40c7 WatchSource:0}: Error finding container acbb9c5d3ad720d95e73ad1a7648a26a4c754d2bf325d71de4d9b14742ae40c7: Status 404 returned error can't find the container with id acbb9c5d3ad720d95e73ad1a7648a26a4c754d2bf325d71de4d9b14742ae40c7
Jul 24 18:15:00.512249 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.512229    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62"]
Jul 24 18:15:00.514036 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.514009    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 18:15:00.533659 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.533637    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn"]
Jul 24 18:15:00.535607 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:15:00.535584    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ece5ed_4bae_4ac3_af68_ee34f3f62a39.slice/crio-4ea28e7f07d5aa59d3dcd9f96b567445f64240df145be0424773f2b8b35370c1 WatchSource:0}: Error finding container 4ea28e7f07d5aa59d3dcd9f96b567445f64240df145be0424773f2b8b35370c1: Status 404 returned error can't find the container with id 4ea28e7f07d5aa59d3dcd9f96b567445f64240df145be0424773f2b8b35370c1
Jul 24 18:15:00.727756 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.727712    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn" event={"ID":"28ece5ed-4bae-4ac3-af68-ee34f3f62a39","Type":"ContainerStarted","Data":"4ea28e7f07d5aa59d3dcd9f96b567445f64240df145be0424773f2b8b35370c1"}
Jul 24 18:15:00.728912 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:00.728870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62" event={"ID":"538b26ac-2494-4c0d-a62d-6e04c3a56bce","Type":"ContainerStarted","Data":"acbb9c5d3ad720d95e73ad1a7648a26a4c754d2bf325d71de4d9b14742ae40c7"}
Jul 24 18:15:03.747884 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:03.747846    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn" event={"ID":"28ece5ed-4bae-4ac3-af68-ee34f3f62a39","Type":"ContainerStarted","Data":"be59e00a860c591a98421e6c72834840e64488fb694f042625afcb1565222543"}
Jul 24 18:15:03.749567 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:03.749541    2562 generic.go:356] "Generic (PLEG): container finished" podID="538b26ac-2494-4c0d-a62d-6e04c3a56bce" containerID="d57013a02e5eb546042e4400f7f99a8bdb1f220216b910684c3921cb5f216096" exitCode=0
Jul 24 18:15:03.749691 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:03.749629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62" event={"ID":"538b26ac-2494-4c0d-a62d-6e04c3a56bce","Type":"ContainerDied","Data":"d57013a02e5eb546042e4400f7f99a8bdb1f220216b910684c3921cb5f216096"}
Jul 24 18:15:03.762407 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:03.762366    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn" podStartSLOduration=1.5713077640000002 podStartE2EDuration="3.762353986s" totalImagesPullingTime="2.191046222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:15:03.76134675 +0000 UTC m=+2110.966038547" watchObservedRunningTime="2026-07-24 18:15:03.762353986 +0000 UTC m=+2110.967045783"
Jul 24 18:15:04.756980 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:04.756925    2562 generic.go:356] "Generic (PLEG): container finished" podID="28ece5ed-4bae-4ac3-af68-ee34f3f62a39" containerID="be59e00a860c591a98421e6c72834840e64488fb694f042625afcb1565222543" exitCode=0
Jul 24 18:15:04.757453 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:04.757019    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn" event={"ID":"28ece5ed-4bae-4ac3-af68-ee34f3f62a39","Type":"ContainerDied","Data":"be59e00a860c591a98421e6c72834840e64488fb694f042625afcb1565222543"}
Jul 24 18:15:04.895728 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:04.895706    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62"
Jul 24 18:15:05.763499 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:05.763431    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62"
Jul 24 18:15:05.763499 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:05.763470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748615-f6z62" event={"ID":"538b26ac-2494-4c0d-a62d-6e04c3a56bce","Type":"ContainerDied","Data":"acbb9c5d3ad720d95e73ad1a7648a26a4c754d2bf325d71de4d9b14742ae40c7"}
Jul 24 18:15:05.764012 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:05.763514    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acbb9c5d3ad720d95e73ad1a7648a26a4c754d2bf325d71de4d9b14742ae40c7"
Jul 24 18:15:05.919161 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:05.919135    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn"
Jul 24 18:15:06.769867 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:06.769838    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn"
Jul 24 18:15:06.769867 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:06.769844    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn" event={"ID":"28ece5ed-4bae-4ac3-af68-ee34f3f62a39","Type":"ContainerDied","Data":"4ea28e7f07d5aa59d3dcd9f96b567445f64240df145be0424773f2b8b35370c1"}
Jul 24 18:15:06.770398 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:06.769903    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ea28e7f07d5aa59d3dcd9f96b567445f64240df145be0424773f2b8b35370c1"
Jul 24 18:15:22.711664 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.711625    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"]
Jul 24 18:15:22.712121 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.712105    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="538b26ac-2494-4c0d-a62d-6e04c3a56bce" containerName="cleanup"
Jul 24 18:15:22.712121 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.712122    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="538b26ac-2494-4c0d-a62d-6e04c3a56bce" containerName="cleanup"
Jul 24 18:15:22.712211 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.712132    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ece5ed-4bae-4ac3-af68-ee34f3f62a39" containerName="cleanup"
Jul 24 18:15:22.712211 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.712139    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ece5ed-4bae-4ac3-af68-ee34f3f62a39" containerName="cleanup"
Jul 24 18:15:22.712211 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.712206    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="538b26ac-2494-4c0d-a62d-6e04c3a56bce" containerName="cleanup"
Jul 24 18:15:22.712318 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.712219    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ece5ed-4bae-4ac3-af68-ee34f3f62a39" containerName="cleanup"
Jul 24 18:15:22.715520 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.715504    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"
Jul 24 18:15:22.718200 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.718174    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:15:22.723050 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.722760    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"]
Jul 24 18:15:22.775224 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.775194    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d\" (UID: \"29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"
Jul 24 18:15:22.875755 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.875725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d\" (UID: \"29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"
Jul 24 18:15:22.876323 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:22.876305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d\" (UID: \"29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"
Jul 24 18:15:23.028238 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:23.028176    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"
Jul 24 18:15:23.373063 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:23.373035    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"]
Jul 24 18:15:23.374101 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:15:23.374076    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29de834e_4c3b_4bc9_a85a_11d9ceb3bb3e.slice/crio-acf6f377409e2665f6fefb687b38b44d0a3b5d63f60ea71b8c0df2376e04c2d7 WatchSource:0}: Error finding container acf6f377409e2665f6fefb687b38b44d0a3b5d63f60ea71b8c0df2376e04c2d7: Status 404 returned error can't find the container with id acf6f377409e2665f6fefb687b38b44d0a3b5d63f60ea71b8c0df2376e04c2d7
Jul 24 18:15:23.873303 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:23.873274    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d" event={"ID":"29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e","Type":"ContainerStarted","Data":"84f22361b4cde3e783a2d483a5cdd875cda74592a7c1fa891072bdc6fe1d5280"}
Jul 24 18:15:23.873733 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:23.873327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d" event={"ID":"29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e","Type":"ContainerStarted","Data":"acf6f377409e2665f6fefb687b38b44d0a3b5d63f60ea71b8c0df2376e04c2d7"}
Jul 24 18:15:23.891213 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:23.891170    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d" podStartSLOduration=1.8911563550000001 podStartE2EDuration="1.891156355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:15:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:15:23.889626766 +0000 UTC m=+2131.094318564" watchObservedRunningTime="2026-07-24 18:15:23.891156355 +0000 UTC m=+2131.095848151"
Jul 24 18:15:24.881593 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:24.881557    2562 generic.go:356] "Generic (PLEG): container finished" podID="29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e" containerID="84f22361b4cde3e783a2d483a5cdd875cda74592a7c1fa891072bdc6fe1d5280" exitCode=0
Jul 24 18:15:24.882015 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:24.881602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d" event={"ID":"29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e","Type":"ContainerDied","Data":"84f22361b4cde3e783a2d483a5cdd875cda74592a7c1fa891072bdc6fe1d5280"}
Jul 24 18:15:26.026625 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:26.026601    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"
Jul 24 18:15:26.097973 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:26.097950    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e-maas-api-service-ca\") pod \"29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e\" (UID: \"29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e\") "
Jul 24 18:15:26.098324 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:26.098303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e" (UID: "29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:15:26.198822 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:26.198772    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e-maas-api-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:15:26.898187 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:26.898155    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"
Jul 24 18:15:26.898187 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:26.898161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d" event={"ID":"29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e","Type":"ContainerDied","Data":"acf6f377409e2665f6fefb687b38b44d0a3b5d63f60ea71b8c0df2376e04c2d7"}
Jul 24 18:15:26.898431 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:26.898203    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acf6f377409e2665f6fefb687b38b44d0a3b5d63f60ea71b8c0df2376e04c2d7"
Jul 24 18:15:38.519718 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:38.519686    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn"]
Jul 24 18:15:38.526543 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:38.526517    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ad8b31a2-29748615-fbhdn"]
Jul 24 18:15:39.408465 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:39.408425    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ece5ed-4bae-4ac3-af68-ee34f3f62a39" path="/var/lib/kubelet/pods/28ece5ed-4bae-4ac3-af68-ee34f3f62a39/volumes"
Jul 24 18:15:42.830972 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:42.830931    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"]
Jul 24 18:15:42.834075 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:42.834051    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad8b31a2-bg96d"]
Jul 24 18:15:43.408437 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:15:43.408407    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e" path="/var/lib/kubelet/pods/29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e/volumes"
Jul 24 18:19:00.484437 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.484406    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"]
Jul 24 18:19:00.484852 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.484842    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e" containerName="revoke-keys"
Jul 24 18:19:00.484926 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.484855    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e" containerName="revoke-keys"
Jul 24 18:19:00.484978 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.484935    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="29de834e-4c3b-4bc9-a85a-11d9ceb3bb3e" containerName="revoke-keys"
Jul 24 18:19:00.487913 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.487897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"
Jul 24 18:19:00.490686 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.490664    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:19:00.496310 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.496281    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"]
Jul 24 18:19:00.617099 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.617072    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab3a8b-8330-49de-8e23-ddb688f2ec33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg\" (UID: \"39ab3a8b-8330-49de-8e23-ddb688f2ec33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"
Jul 24 18:19:00.717948 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.717923    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab3a8b-8330-49de-8e23-ddb688f2ec33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg\" (UID: \"39ab3a8b-8330-49de-8e23-ddb688f2ec33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"
Jul 24 18:19:00.718528 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.718510    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab3a8b-8330-49de-8e23-ddb688f2ec33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg\" (UID: \"39ab3a8b-8330-49de-8e23-ddb688f2ec33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"
Jul 24 18:19:00.799980 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:00.799956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"
Jul 24 18:19:01.148606 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:01.148576    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"]
Jul 24 18:19:01.149912 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:19:01.149860    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ab3a8b_8330_49de_8e23_ddb688f2ec33.slice/crio-fa8a44f0a6375bc88813c09bf237ab0418e6be4ab35026e163b35d3a68f1286c WatchSource:0}: Error finding container fa8a44f0a6375bc88813c09bf237ab0418e6be4ab35026e163b35d3a68f1286c: Status 404 returned error can't find the container with id fa8a44f0a6375bc88813c09bf237ab0418e6be4ab35026e163b35d3a68f1286c
Jul 24 18:19:01.207539 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:01.207514    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg" event={"ID":"39ab3a8b-8330-49de-8e23-ddb688f2ec33","Type":"ContainerStarted","Data":"fa8a44f0a6375bc88813c09bf237ab0418e6be4ab35026e163b35d3a68f1286c"}
Jul 24 18:19:02.214747 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:02.214682    2562 generic.go:356] "Generic (PLEG): container finished" podID="39ab3a8b-8330-49de-8e23-ddb688f2ec33" containerID="64d0c97752555f5e4f4479e542c602785319d34aae24ca6fe3693acfec974046" exitCode=0
Jul 24 18:19:02.215083 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:02.214761    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg" event={"ID":"39ab3a8b-8330-49de-8e23-ddb688f2ec33","Type":"ContainerDied","Data":"64d0c97752555f5e4f4479e542c602785319d34aae24ca6fe3693acfec974046"}
Jul 24 18:19:03.357587 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:03.357565    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"
Jul 24 18:19:03.440483 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:03.440459    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab3a8b-8330-49de-8e23-ddb688f2ec33-maas-api-service-ca\") pod \"39ab3a8b-8330-49de-8e23-ddb688f2ec33\" (UID: \"39ab3a8b-8330-49de-8e23-ddb688f2ec33\") "
Jul 24 18:19:03.440772 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:03.440751    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39ab3a8b-8330-49de-8e23-ddb688f2ec33-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "39ab3a8b-8330-49de-8e23-ddb688f2ec33" (UID: "39ab3a8b-8330-49de-8e23-ddb688f2ec33"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:19:03.541597 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:03.541573    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab3a8b-8330-49de-8e23-ddb688f2ec33-maas-api-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:19:04.227737 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:04.227707    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"
Jul 24 18:19:04.227985 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:04.227741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg" event={"ID":"39ab3a8b-8330-49de-8e23-ddb688f2ec33","Type":"ContainerDied","Data":"fa8a44f0a6375bc88813c09bf237ab0418e6be4ab35026e163b35d3a68f1286c"}
Jul 24 18:19:04.227985 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:04.227790    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa8a44f0a6375bc88813c09bf237ab0418e6be4ab35026e163b35d3a68f1286c"
Jul 24 18:19:15.586979 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:15.586926    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"]
Jul 24 18:19:15.589664 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:15.589635    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6e3964-bhxrg"]
Jul 24 18:19:17.407832 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:17.407783    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39ab3a8b-8330-49de-8e23-ddb688f2ec33" path="/var/lib/kubelet/pods/39ab3a8b-8330-49de-8e23-ddb688f2ec33/volumes"
Jul 24 18:19:22.555377 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.555342    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"]
Jul 24 18:19:22.555803 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.555757    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ab3a8b-8330-49de-8e23-ddb688f2ec33" containerName="revoke-keys"
Jul 24 18:19:22.555803 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.555768    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ab3a8b-8330-49de-8e23-ddb688f2ec33" containerName="revoke-keys"
Jul 24 18:19:22.555901 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.555834    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ab3a8b-8330-49de-8e23-ddb688f2ec33" containerName="revoke-keys"
Jul 24 18:19:22.558969 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.558950    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"
Jul 24 18:19:22.561627 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.561603    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:19:22.570001 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.569973    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"]
Jul 24 18:19:22.585945 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.585922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d4f75e7-e805-4818-9bea-c7a66a769529-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk\" (UID: \"5d4f75e7-e805-4818-9bea-c7a66a769529\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"
Jul 24 18:19:22.686530 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.686509    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d4f75e7-e805-4818-9bea-c7a66a769529-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk\" (UID: \"5d4f75e7-e805-4818-9bea-c7a66a769529\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"
Jul 24 18:19:22.687063 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.687047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d4f75e7-e805-4818-9bea-c7a66a769529-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk\" (UID: \"5d4f75e7-e805-4818-9bea-c7a66a769529\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"
Jul 24 18:19:22.870414 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:22.870358    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"
Jul 24 18:19:23.211869 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:23.211840    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"]
Jul 24 18:19:23.212382 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:19:23.212361    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d4f75e7_e805_4818_9bea_c7a66a769529.slice/crio-ce9e09af239f56bc82efd4e7bdb6824abdd9aa6152dd549d7cb7db38fb45ac94 WatchSource:0}: Error finding container ce9e09af239f56bc82efd4e7bdb6824abdd9aa6152dd549d7cb7db38fb45ac94: Status 404 returned error can't find the container with id ce9e09af239f56bc82efd4e7bdb6824abdd9aa6152dd549d7cb7db38fb45ac94
Jul 24 18:19:23.345835 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:23.345809    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk" event={"ID":"5d4f75e7-e805-4818-9bea-c7a66a769529","Type":"ContainerStarted","Data":"ce9e09af239f56bc82efd4e7bdb6824abdd9aa6152dd549d7cb7db38fb45ac94"}
Jul 24 18:19:24.352831 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:24.352801    2562 generic.go:356] "Generic (PLEG): container finished" podID="5d4f75e7-e805-4818-9bea-c7a66a769529" containerID="52072f3c9687eeb0a9eea9e7d05c259ff1ceedc1ed12f8a6d44e34cab6bb894f" exitCode=0
Jul 24 18:19:24.353208 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:24.352869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk" event={"ID":"5d4f75e7-e805-4818-9bea-c7a66a769529","Type":"ContainerDied","Data":"52072f3c9687eeb0a9eea9e7d05c259ff1ceedc1ed12f8a6d44e34cab6bb894f"}
Jul 24 18:19:25.500289 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:25.500264    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"
Jul 24 18:19:25.505310 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:25.505293    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d4f75e7-e805-4818-9bea-c7a66a769529-maas-api-service-ca\") pod \"5d4f75e7-e805-4818-9bea-c7a66a769529\" (UID: \"5d4f75e7-e805-4818-9bea-c7a66a769529\") "
Jul 24 18:19:25.505647 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:25.505617    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d4f75e7-e805-4818-9bea-c7a66a769529-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d4f75e7-e805-4818-9bea-c7a66a769529" (UID: "5d4f75e7-e805-4818-9bea-c7a66a769529"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:19:25.606117 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:25.606089    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d4f75e7-e805-4818-9bea-c7a66a769529-maas-api-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:19:26.368028 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:26.367977    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk" event={"ID":"5d4f75e7-e805-4818-9bea-c7a66a769529","Type":"ContainerDied","Data":"ce9e09af239f56bc82efd4e7bdb6824abdd9aa6152dd549d7cb7db38fb45ac94"}
Jul 24 18:19:26.368221 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:26.368036    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"
Jul 24 18:19:26.368221 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:26.368085    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce9e09af239f56bc82efd4e7bdb6824abdd9aa6152dd549d7cb7db38fb45ac94"
Jul 24 18:19:37.695601 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:37.695568    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"]
Jul 24 18:19:37.702062 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:37.702034    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f1d5b-kxrkk"]
Jul 24 18:19:39.407794 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:39.407757    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d4f75e7-e805-4818-9bea-c7a66a769529" path="/var/lib/kubelet/pods/5d4f75e7-e805-4818-9bea-c7a66a769529/volumes"
Jul 24 18:19:53.490199 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:53.490174    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:19:53.498667 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:53.498645    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:19:53.504211 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:53.504187    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:19:53.511703 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:53.511685    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:19:57.533916 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.533865    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"]
Jul 24 18:19:57.534513 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.534489    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d4f75e7-e805-4818-9bea-c7a66a769529" containerName="revoke-keys"
Jul 24 18:19:57.534513 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.534514    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d4f75e7-e805-4818-9bea-c7a66a769529" containerName="revoke-keys"
Jul 24 18:19:57.534676 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.534593    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d4f75e7-e805-4818-9bea-c7a66a769529" containerName="revoke-keys"
Jul 24 18:19:57.538791 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.538776    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:19:57.541489 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.541469    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d1a9df\""
Jul 24 18:19:57.541575 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.541469    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-d1a9df-dockercfg-vx6s9\""
Jul 24 18:19:57.557279 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.557256    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"]
Jul 24 18:19:57.675147 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.675115    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ee1126-7413-4f05-9232-88d59d1432e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5\" (UID: \"07ee1126-7413-4f05-9232-88d59d1432e2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:19:57.675304 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.675277    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6kpp\" (UniqueName: \"kubernetes.io/projected/07ee1126-7413-4f05-9232-88d59d1432e2-kube-api-access-h6kpp\") pod \"payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5\" (UID: \"07ee1126-7413-4f05-9232-88d59d1432e2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:19:57.776699 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.776667    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6kpp\" (UniqueName: \"kubernetes.io/projected/07ee1126-7413-4f05-9232-88d59d1432e2-kube-api-access-h6kpp\") pod \"payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5\" (UID: \"07ee1126-7413-4f05-9232-88d59d1432e2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:19:57.776699 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.776702    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ee1126-7413-4f05-9232-88d59d1432e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5\" (UID: \"07ee1126-7413-4f05-9232-88d59d1432e2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:19:57.777987 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.777956    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ee1126-7413-4f05-9232-88d59d1432e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5\" (UID: \"07ee1126-7413-4f05-9232-88d59d1432e2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:19:57.786672 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.786647    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6kpp\" (UniqueName: \"kubernetes.io/projected/07ee1126-7413-4f05-9232-88d59d1432e2-kube-api-access-h6kpp\") pod \"payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5\" (UID: \"07ee1126-7413-4f05-9232-88d59d1432e2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:19:57.849992 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:57.849970    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:19:58.199832 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:58.199807    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"]
Jul 24 18:19:58.200412 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:19:58.200391    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ee1126_7413_4f05_9232_88d59d1432e2.slice/crio-210cca8f91a42e7e9c6ba9f1fed7a9dd1456fb2825712ed42141674ad2a5744b WatchSource:0}: Error finding container 210cca8f91a42e7e9c6ba9f1fed7a9dd1456fb2825712ed42141674ad2a5744b: Status 404 returned error can't find the container with id 210cca8f91a42e7e9c6ba9f1fed7a9dd1456fb2825712ed42141674ad2a5744b
Jul 24 18:19:58.558305 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:58.558270    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5" event={"ID":"07ee1126-7413-4f05-9232-88d59d1432e2","Type":"ContainerStarted","Data":"f520dd7d5671fc9acc2b8e96e0562b49340a3791f3d5c44553c1980470c75442"}
Jul 24 18:19:58.558727 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:58.558327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5" event={"ID":"07ee1126-7413-4f05-9232-88d59d1432e2","Type":"ContainerStarted","Data":"210cca8f91a42e7e9c6ba9f1fed7a9dd1456fb2825712ed42141674ad2a5744b"}
Jul 24 18:19:58.558727 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:58.558362    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:19:58.581416 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:19:58.581372    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5" podStartSLOduration=1.5813568569999998 podStartE2EDuration="1.581356857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:19:58.578489899 +0000 UTC m=+2405.783181696" watchObservedRunningTime="2026-07-24 18:19:58.581356857 +0000 UTC m=+2405.786048655"
Jul 24 18:20:09.190575 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.190536    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"]
Jul 24 18:20:09.194552 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.194529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"
Jul 24 18:20:09.197407 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.197377    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:20:09.202928 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.202686    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"]
Jul 24 18:20:09.269255 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.269225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93419f7-1612-4cbb-8644-ac06b3b6ae9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-013528-nhmvm\" (UID: \"a93419f7-1612-4cbb-8644-ac06b3b6ae9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"
Jul 24 18:20:09.369617 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.369588    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93419f7-1612-4cbb-8644-ac06b3b6ae9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-013528-nhmvm\" (UID: \"a93419f7-1612-4cbb-8644-ac06b3b6ae9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"
Jul 24 18:20:09.370209 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.370189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93419f7-1612-4cbb-8644-ac06b3b6ae9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-013528-nhmvm\" (UID: \"a93419f7-1612-4cbb-8644-ac06b3b6ae9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"
Jul 24 18:20:09.506702 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.506640    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"
Jul 24 18:20:09.566675 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.566651    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:20:09.860501 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:09.860477    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"]
Jul 24 18:20:09.862563 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:20:09.862536    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93419f7_1612_4cbb_8644_ac06b3b6ae9a.slice/crio-b07851cb1353c8da109c833f4f3cab37c5ad9fa604829505420e55310e3e5a3c WatchSource:0}: Error finding container b07851cb1353c8da109c833f4f3cab37c5ad9fa604829505420e55310e3e5a3c: Status 404 returned error can't find the container with id b07851cb1353c8da109c833f4f3cab37c5ad9fa604829505420e55310e3e5a3c
Jul 24 18:20:10.631485 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:10.631457    2562 generic.go:356] "Generic (PLEG): container finished" podID="a93419f7-1612-4cbb-8644-ac06b3b6ae9a" containerID="c70bb46927caea4eeccc9d837efaa8e89496c635fc3095ee833ae3cd85550d6a" exitCode=0
Jul 24 18:20:10.631849 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:10.631496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm" event={"ID":"a93419f7-1612-4cbb-8644-ac06b3b6ae9a","Type":"ContainerDied","Data":"c70bb46927caea4eeccc9d837efaa8e89496c635fc3095ee833ae3cd85550d6a"}
Jul 24 18:20:10.631849 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:10.631532    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm" event={"ID":"a93419f7-1612-4cbb-8644-ac06b3b6ae9a","Type":"ContainerStarted","Data":"b07851cb1353c8da109c833f4f3cab37c5ad9fa604829505420e55310e3e5a3c"}
Jul 24 18:20:11.778594 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:11.778566    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"
Jul 24 18:20:11.892326 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:11.892262    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93419f7-1612-4cbb-8644-ac06b3b6ae9a-maas-api-service-ca\") pod \"a93419f7-1612-4cbb-8644-ac06b3b6ae9a\" (UID: \"a93419f7-1612-4cbb-8644-ac06b3b6ae9a\") "
Jul 24 18:20:11.892578 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:11.892557    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a93419f7-1612-4cbb-8644-ac06b3b6ae9a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a93419f7-1612-4cbb-8644-ac06b3b6ae9a" (UID: "a93419f7-1612-4cbb-8644-ac06b3b6ae9a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:20:11.993644 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:11.993621    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93419f7-1612-4cbb-8644-ac06b3b6ae9a-maas-api-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:20:12.644357 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:12.644322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm" event={"ID":"a93419f7-1612-4cbb-8644-ac06b3b6ae9a","Type":"ContainerDied","Data":"b07851cb1353c8da109c833f4f3cab37c5ad9fa604829505420e55310e3e5a3c"}
Jul 24 18:20:12.644357 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:12.644341    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"
Jul 24 18:20:12.644558 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:12.644377    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b07851cb1353c8da109c833f4f3cab37c5ad9fa604829505420e55310e3e5a3c"
Jul 24 18:20:29.288590 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:29.288506    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"]
Jul 24 18:20:29.292856 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:29.292830    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-013528-nhmvm"]
Jul 24 18:20:29.407826 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:29.407792    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a93419f7-1612-4cbb-8644-ac06b3b6ae9a" path="/var/lib/kubelet/pods/a93419f7-1612-4cbb-8644-ac06b3b6ae9a/volumes"
Jul 24 18:20:36.240255 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.240217    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"]
Jul 24 18:20:36.240738 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.240700    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93419f7-1612-4cbb-8644-ac06b3b6ae9a" containerName="revoke-keys"
Jul 24 18:20:36.240738 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.240713    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93419f7-1612-4cbb-8644-ac06b3b6ae9a" containerName="revoke-keys"
Jul 24 18:20:36.240816 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.240778    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93419f7-1612-4cbb-8644-ac06b3b6ae9a" containerName="revoke-keys"
Jul 24 18:20:36.244159 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.244143    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"
Jul 24 18:20:36.248148 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.248128    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:20:36.253530 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.253482    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"]
Jul 24 18:20:36.282868 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.282831    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac42afe3-482c-44dd-beec-9bf79d9a6b80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq\" (UID: \"ac42afe3-482c-44dd-beec-9bf79d9a6b80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"
Jul 24 18:20:36.383563 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.383538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac42afe3-482c-44dd-beec-9bf79d9a6b80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq\" (UID: \"ac42afe3-482c-44dd-beec-9bf79d9a6b80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"
Jul 24 18:20:36.384217 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.384194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac42afe3-482c-44dd-beec-9bf79d9a6b80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq\" (UID: \"ac42afe3-482c-44dd-beec-9bf79d9a6b80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"
Jul 24 18:20:36.555248 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.555220    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"
Jul 24 18:20:36.694127 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.694100    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"]
Jul 24 18:20:36.696115 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:20:36.696082    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac42afe3_482c_44dd_beec_9bf79d9a6b80.slice/crio-a8b62341986e147d248019b48c0d4cc1f9af6227f02471d1dc685f0c32052821 WatchSource:0}: Error finding container a8b62341986e147d248019b48c0d4cc1f9af6227f02471d1dc685f0c32052821: Status 404 returned error can't find the container with id a8b62341986e147d248019b48c0d4cc1f9af6227f02471d1dc685f0c32052821
Jul 24 18:20:36.789565 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:36.789537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq" event={"ID":"ac42afe3-482c-44dd-beec-9bf79d9a6b80","Type":"ContainerStarted","Data":"a8b62341986e147d248019b48c0d4cc1f9af6227f02471d1dc685f0c32052821"}
Jul 24 18:20:37.797044 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:37.797013    2562 generic.go:356] "Generic (PLEG): container finished" podID="ac42afe3-482c-44dd-beec-9bf79d9a6b80" containerID="895278b9f3baab922ed68639123899a7593fe01e6c5e9e8d36e29b121b011303" exitCode=0
Jul 24 18:20:37.797401 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:37.797074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq" event={"ID":"ac42afe3-482c-44dd-beec-9bf79d9a6b80","Type":"ContainerDied","Data":"895278b9f3baab922ed68639123899a7593fe01e6c5e9e8d36e29b121b011303"}
Jul 24 18:20:38.949997 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:38.949976    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"
Jul 24 18:20:39.006115 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:39.006087    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac42afe3-482c-44dd-beec-9bf79d9a6b80-maas-api-service-ca\") pod \"ac42afe3-482c-44dd-beec-9bf79d9a6b80\" (UID: \"ac42afe3-482c-44dd-beec-9bf79d9a6b80\") "
Jul 24 18:20:39.006454 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:39.006431    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac42afe3-482c-44dd-beec-9bf79d9a6b80-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ac42afe3-482c-44dd-beec-9bf79d9a6b80" (UID: "ac42afe3-482c-44dd-beec-9bf79d9a6b80"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:20:39.107654 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:39.107591    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac42afe3-482c-44dd-beec-9bf79d9a6b80-maas-api-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:20:39.809810 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:39.809784    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"
Jul 24 18:20:39.809977 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:39.809783    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq" event={"ID":"ac42afe3-482c-44dd-beec-9bf79d9a6b80","Type":"ContainerDied","Data":"a8b62341986e147d248019b48c0d4cc1f9af6227f02471d1dc685f0c32052821"}
Jul 24 18:20:39.809977 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:39.809929    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8b62341986e147d248019b48c0d4cc1f9af6227f02471d1dc685f0c32052821"
Jul 24 18:20:51.770698 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:51.770654    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"]
Jul 24 18:20:51.771186 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:51.770936    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5" podUID="07ee1126-7413-4f05-9232-88d59d1432e2" containerName="payload-pre-processing" containerID="cri-o://f520dd7d5671fc9acc2b8e96e0562b49340a3791f3d5c44553c1980470c75442" gracePeriod=30
Jul 24 18:20:51.887604 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:51.887572    2562 generic.go:356] "Generic (PLEG): container finished" podID="07ee1126-7413-4f05-9232-88d59d1432e2" containerID="f520dd7d5671fc9acc2b8e96e0562b49340a3791f3d5c44553c1980470c75442" exitCode=0
Jul 24 18:20:51.887748 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:51.887644    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5" event={"ID":"07ee1126-7413-4f05-9232-88d59d1432e2","Type":"ContainerDied","Data":"f520dd7d5671fc9acc2b8e96e0562b49340a3791f3d5c44553c1980470c75442"}
Jul 24 18:20:52.032922 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.032856    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:20:52.108162 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.108136    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ee1126-7413-4f05-9232-88d59d1432e2-plugins-config-volume\") pod \"07ee1126-7413-4f05-9232-88d59d1432e2\" (UID: \"07ee1126-7413-4f05-9232-88d59d1432e2\") "
Jul 24 18:20:52.108290 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.108207    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6kpp\" (UniqueName: \"kubernetes.io/projected/07ee1126-7413-4f05-9232-88d59d1432e2-kube-api-access-h6kpp\") pod \"07ee1126-7413-4f05-9232-88d59d1432e2\" (UID: \"07ee1126-7413-4f05-9232-88d59d1432e2\") "
Jul 24 18:20:52.108541 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.108519    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07ee1126-7413-4f05-9232-88d59d1432e2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "07ee1126-7413-4f05-9232-88d59d1432e2" (UID: "07ee1126-7413-4f05-9232-88d59d1432e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:20:52.110236 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.110214    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ee1126-7413-4f05-9232-88d59d1432e2-kube-api-access-h6kpp" (OuterVolumeSpecName: "kube-api-access-h6kpp") pod "07ee1126-7413-4f05-9232-88d59d1432e2" (UID: "07ee1126-7413-4f05-9232-88d59d1432e2"). InnerVolumeSpecName "kube-api-access-h6kpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:20:52.209356 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.209328    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6kpp\" (UniqueName: \"kubernetes.io/projected/07ee1126-7413-4f05-9232-88d59d1432e2-kube-api-access-h6kpp\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:20:52.209356 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.209350    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ee1126-7413-4f05-9232-88d59d1432e2-plugins-config-volume\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:20:52.895024 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.894997    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"
Jul 24 18:20:52.895024 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.895009    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5" event={"ID":"07ee1126-7413-4f05-9232-88d59d1432e2","Type":"ContainerDied","Data":"210cca8f91a42e7e9c6ba9f1fed7a9dd1456fb2825712ed42141674ad2a5744b"}
Jul 24 18:20:52.895576 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.895079    2562 scope.go:117] "RemoveContainer" containerID="f520dd7d5671fc9acc2b8e96e0562b49340a3791f3d5c44553c1980470c75442"
Jul 24 18:20:52.917488 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.917467    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"]
Jul 24 18:20:52.924887 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:52.924855    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d1a9df-747fd5bc44-g45r5"]
Jul 24 18:20:53.409338 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:53.409304    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07ee1126-7413-4f05-9232-88d59d1432e2" path="/var/lib/kubelet/pods/07ee1126-7413-4f05-9232-88d59d1432e2/volumes"
Jul 24 18:20:56.306688 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:56.306652    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"]
Jul 24 18:20:56.309277 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:56.309245    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d1a9df-8tgxq"]
Jul 24 18:20:57.407785 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:20:57.407751    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac42afe3-482c-44dd-beec-9bf79d9a6b80" path="/var/lib/kubelet/pods/ac42afe3-482c-44dd-beec-9bf79d9a6b80/volumes"
Jul 24 18:21:53.744047 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:21:53.744015    2562 scope.go:117] "RemoveContainer" containerID="84f22361b4cde3e783a2d483a5cdd875cda74592a7c1fa891072bdc6fe1d5280"
Jul 24 18:21:53.752975 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:21:53.752954    2562 scope.go:117] "RemoveContainer" containerID="be59e00a860c591a98421e6c72834840e64488fb694f042625afcb1565222543"
Jul 24 18:24:53.535077 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:24:53.535047    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:24:53.544895 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:24:53.544855    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:24:53.550073 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:24:53.550021    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:24:53.558716 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:24:53.558700    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:25:53.807616 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:25:53.807580    2562 scope.go:117] "RemoveContainer" containerID="52072f3c9687eeb0a9eea9e7d05c259ff1ceedc1ed12f8a6d44e34cab6bb894f"
Jul 24 18:25:53.816679 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:25:53.816658    2562 scope.go:117] "RemoveContainer" containerID="64d0c97752555f5e4f4479e542c602785319d34aae24ca6fe3693acfec974046"
Jul 24 18:26:53.842360 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:26:53.842327    2562 scope.go:117] "RemoveContainer" containerID="895278b9f3baab922ed68639123899a7593fe01e6c5e9e8d36e29b121b011303"
Jul 24 18:26:53.852536 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:26:53.852512    2562 scope.go:117] "RemoveContainer" containerID="c70bb46927caea4eeccc9d837efaa8e89496c635fc3095ee833ae3cd85550d6a"
Jul 24 18:27:17.011672 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.011599    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-140601033825456"]
Jul 24 18:27:17.012038 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.012004    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac42afe3-482c-44dd-beec-9bf79d9a6b80" containerName="revoke-keys"
Jul 24 18:27:17.012038 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.012017    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac42afe3-482c-44dd-beec-9bf79d9a6b80" containerName="revoke-keys"
Jul 24 18:27:17.012038 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.012035    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ee1126-7413-4f05-9232-88d59d1432e2" containerName="payload-pre-processing"
Jul 24 18:27:17.012038 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.012041    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ee1126-7413-4f05-9232-88d59d1432e2" containerName="payload-pre-processing"
Jul 24 18:27:17.012178 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.012113    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ee1126-7413-4f05-9232-88d59d1432e2" containerName="payload-pre-processing"
Jul 24 18:27:17.012178 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.012122    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac42afe3-482c-44dd-beec-9bf79d9a6b80" containerName="revoke-keys"
Jul 24 18:27:17.015140 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.015125    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033825456"
Jul 24 18:27:17.018396 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.018377    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mzggf\""
Jul 24 18:27:17.028001 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.027978    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601033825456"]
Jul 24 18:27:17.096679 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.096657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7q9c\" (UniqueName: \"kubernetes.io/projected/c8437a88-a5d1-4bc4-8d4c-47797e62d71a-kube-api-access-r7q9c\") pod \"test-curl-7651-140601033825456\" (UID: \"c8437a88-a5d1-4bc4-8d4c-47797e62d71a\") " pod="opendatahub/test-curl-7651-140601033825456"
Jul 24 18:27:17.197614 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.197580    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7q9c\" (UniqueName: \"kubernetes.io/projected/c8437a88-a5d1-4bc4-8d4c-47797e62d71a-kube-api-access-r7q9c\") pod \"test-curl-7651-140601033825456\" (UID: \"c8437a88-a5d1-4bc4-8d4c-47797e62d71a\") " pod="opendatahub/test-curl-7651-140601033825456"
Jul 24 18:27:17.207064 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.207041    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7q9c\" (UniqueName: \"kubernetes.io/projected/c8437a88-a5d1-4bc4-8d4c-47797e62d71a-kube-api-access-r7q9c\") pod \"test-curl-7651-140601033825456\" (UID: \"c8437a88-a5d1-4bc4-8d4c-47797e62d71a\") " pod="opendatahub/test-curl-7651-140601033825456"
Jul 24 18:27:17.324117 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.324098    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033825456"
Jul 24 18:27:17.458029 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.458004    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601033825456"]
Jul 24 18:27:17.459635 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:27:17.459613    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8437a88_a5d1_4bc4_8d4c_47797e62d71a.slice/crio-55b9b5214c88f3f65fdda1a5daeacbfd5c0c8ae68d916d06dadf7bfa9e605e0c WatchSource:0}: Error finding container 55b9b5214c88f3f65fdda1a5daeacbfd5c0c8ae68d916d06dadf7bfa9e605e0c: Status 404 returned error can't find the container with id 55b9b5214c88f3f65fdda1a5daeacbfd5c0c8ae68d916d06dadf7bfa9e605e0c
Jul 24 18:27:17.460850 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:17.460834    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 18:27:18.245687 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:18.245662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601033825456" event={"ID":"c8437a88-a5d1-4bc4-8d4c-47797e62d71a","Type":"ContainerStarted","Data":"5de9c44109c021c67798e4766fcddee7d44bda693be7e47c48521df7bc44391d"}
Jul 24 18:27:18.246047 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:18.245700    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601033825456" event={"ID":"c8437a88-a5d1-4bc4-8d4c-47797e62d71a","Type":"ContainerStarted","Data":"55b9b5214c88f3f65fdda1a5daeacbfd5c0c8ae68d916d06dadf7bfa9e605e0c"}
Jul 24 18:27:19.253094 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.253063    2562 generic.go:356] "Generic (PLEG): container finished" podID="c8437a88-a5d1-4bc4-8d4c-47797e62d71a" containerID="5de9c44109c021c67798e4766fcddee7d44bda693be7e47c48521df7bc44391d" exitCode=0
Jul 24 18:27:19.253503 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.253156    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601033825456" event={"ID":"c8437a88-a5d1-4bc4-8d4c-47797e62d71a","Type":"ContainerDied","Data":"5de9c44109c021c67798e4766fcddee7d44bda693be7e47c48521df7bc44391d"}
Jul 24 18:27:19.383953 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.383931    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033825456"
Jul 24 18:27:19.415724 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.415699    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7q9c\" (UniqueName: \"kubernetes.io/projected/c8437a88-a5d1-4bc4-8d4c-47797e62d71a-kube-api-access-r7q9c\") pod \"c8437a88-a5d1-4bc4-8d4c-47797e62d71a\" (UID: \"c8437a88-a5d1-4bc4-8d4c-47797e62d71a\") "
Jul 24 18:27:19.417633 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.417606    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8437a88-a5d1-4bc4-8d4c-47797e62d71a-kube-api-access-r7q9c" (OuterVolumeSpecName: "kube-api-access-r7q9c") pod "c8437a88-a5d1-4bc4-8d4c-47797e62d71a" (UID: "c8437a88-a5d1-4bc4-8d4c-47797e62d71a"). InnerVolumeSpecName "kube-api-access-r7q9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:27:19.424441 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.424420    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7651-140601033825456_c8437a88-a5d1-4bc4-8d4c-47797e62d71a/test-curl-7651-140601033825456/0.log"
Jul 24 18:27:19.462690 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.462661    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-140601033825456"]
Jul 24 18:27:19.470254 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.470234    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-140601033825456"]
Jul 24 18:27:19.516346 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.516298    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7q9c\" (UniqueName: \"kubernetes.io/projected/c8437a88-a5d1-4bc4-8d4c-47797e62d71a-kube-api-access-r7q9c\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:27:19.593357 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.593337    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-140601033885616"]
Jul 24 18:27:19.593751 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.593738    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8437a88-a5d1-4bc4-8d4c-47797e62d71a" containerName="test-curl-7651-140601033825456"
Jul 24 18:27:19.593796 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.593752    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8437a88-a5d1-4bc4-8d4c-47797e62d71a" containerName="test-curl-7651-140601033825456"
Jul 24 18:27:19.593835 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.593831    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8437a88-a5d1-4bc4-8d4c-47797e62d71a" containerName="test-curl-7651-140601033825456"
Jul 24 18:27:19.596929 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.596914    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033885616"
Jul 24 18:27:19.606770 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.606746    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601033885616"]
Jul 24 18:27:19.617612 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.617587    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgf6s\" (UniqueName: \"kubernetes.io/projected/65d6bab7-e06d-4834-bfd4-5fe069b02867-kube-api-access-cgf6s\") pod \"test-curl-7651-140601033885616\" (UID: \"65d6bab7-e06d-4834-bfd4-5fe069b02867\") " pod="opendatahub/test-curl-7651-140601033885616"
Jul 24 18:27:19.718179 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.718160    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgf6s\" (UniqueName: \"kubernetes.io/projected/65d6bab7-e06d-4834-bfd4-5fe069b02867-kube-api-access-cgf6s\") pod \"test-curl-7651-140601033885616\" (UID: \"65d6bab7-e06d-4834-bfd4-5fe069b02867\") " pod="opendatahub/test-curl-7651-140601033885616"
Jul 24 18:27:19.727671 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.727646    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgf6s\" (UniqueName: \"kubernetes.io/projected/65d6bab7-e06d-4834-bfd4-5fe069b02867-kube-api-access-cgf6s\") pod \"test-curl-7651-140601033885616\" (UID: \"65d6bab7-e06d-4834-bfd4-5fe069b02867\") " pod="opendatahub/test-curl-7651-140601033885616"
Jul 24 18:27:19.906705 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:19.906680    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033885616"
Jul 24 18:27:20.040735 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:20.040701    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601033885616"]
Jul 24 18:27:20.042221 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:27:20.042201    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65d6bab7_e06d_4834_bfd4_5fe069b02867.slice/crio-6106a889f5162ffa4dd36dbaa4c7ebf0bc839a8b9de43f9cd165b0dad82d2d90 WatchSource:0}: Error finding container 6106a889f5162ffa4dd36dbaa4c7ebf0bc839a8b9de43f9cd165b0dad82d2d90: Status 404 returned error can't find the container with id 6106a889f5162ffa4dd36dbaa4c7ebf0bc839a8b9de43f9cd165b0dad82d2d90
Jul 24 18:27:20.259755 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:20.259686    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601033885616" event={"ID":"65d6bab7-e06d-4834-bfd4-5fe069b02867","Type":"ContainerStarted","Data":"6106a889f5162ffa4dd36dbaa4c7ebf0bc839a8b9de43f9cd165b0dad82d2d90"}
Jul 24 18:27:20.260799 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:20.260782    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55b9b5214c88f3f65fdda1a5daeacbfd5c0c8ae68d916d06dadf7bfa9e605e0c"
Jul 24 18:27:20.260799 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:20.260785    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033825456"
Jul 24 18:27:20.271661 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:27:20.271639    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-140601033825456\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="c8437a88-a5d1-4bc4-8d4c-47797e62d71a" pod="opendatahub/test-curl-7651-140601033825456"
Jul 24 18:27:21.268232 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:21.268197    2562 generic.go:356] "Generic (PLEG): container finished" podID="65d6bab7-e06d-4834-bfd4-5fe069b02867" containerID="b8f7fdc06805665520bde67d52a33f49db6742e157aad19ff7073b3a53cdcc81" exitCode=0
Jul 24 18:27:21.268600 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:21.268286    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601033885616" event={"ID":"65d6bab7-e06d-4834-bfd4-5fe069b02867","Type":"ContainerDied","Data":"b8f7fdc06805665520bde67d52a33f49db6742e157aad19ff7073b3a53cdcc81"}
Jul 24 18:27:21.287390 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:27:21.287358    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-140601033825456\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="c8437a88-a5d1-4bc4-8d4c-47797e62d71a" pod="opendatahub/test-curl-7651-140601033825456"
Jul 24 18:27:21.407369 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:21.407345    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8437a88-a5d1-4bc4-8d4c-47797e62d71a" path="/var/lib/kubelet/pods/c8437a88-a5d1-4bc4-8d4c-47797e62d71a/volumes"
Jul 24 18:27:22.418057 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.418036    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033885616"
Jul 24 18:27:22.439749 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.439725    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7651-140601033885616_65d6bab7-e06d-4834-bfd4-5fe069b02867/test-curl-7651-140601033885616/0.log"
Jul 24 18:27:22.440907 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.440862    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgf6s\" (UniqueName: \"kubernetes.io/projected/65d6bab7-e06d-4834-bfd4-5fe069b02867-kube-api-access-cgf6s\") pod \"65d6bab7-e06d-4834-bfd4-5fe069b02867\" (UID: \"65d6bab7-e06d-4834-bfd4-5fe069b02867\") "
Jul 24 18:27:22.442873 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.442839    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65d6bab7-e06d-4834-bfd4-5fe069b02867-kube-api-access-cgf6s" (OuterVolumeSpecName: "kube-api-access-cgf6s") pod "65d6bab7-e06d-4834-bfd4-5fe069b02867" (UID: "65d6bab7-e06d-4834-bfd4-5fe069b02867"). InnerVolumeSpecName "kube-api-access-cgf6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:27:22.464431 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.464408    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-140601033885616"]
Jul 24 18:27:22.469737 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.469718    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-140601033885616"]
Jul 24 18:27:22.541806 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.541786    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgf6s\" (UniqueName: \"kubernetes.io/projected/65d6bab7-e06d-4834-bfd4-5fe069b02867-kube-api-access-cgf6s\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:27:22.643123 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.643099    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:22.643507 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.643494    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d6bab7-e06d-4834-bfd4-5fe069b02867" containerName="test-curl-7651-140601033885616"
Jul 24 18:27:22.643562 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.643508    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d6bab7-e06d-4834-bfd4-5fe069b02867" containerName="test-curl-7651-140601033885616"
Jul 24 18:27:22.643603 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.643591    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d6bab7-e06d-4834-bfd4-5fe069b02867" containerName="test-curl-7651-140601033885616"
Jul 24 18:27:22.647784 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.647760    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:22.656457 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.656429    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:22.743454 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.743433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk5h4\" (UniqueName: \"kubernetes.io/projected/424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c-kube-api-access-gk5h4\") pod \"test-curl-7651-140601040222224\" (UID: \"424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c\") " pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:22.844613 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.844554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk5h4\" (UniqueName: \"kubernetes.io/projected/424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c-kube-api-access-gk5h4\") pod \"test-curl-7651-140601040222224\" (UID: \"424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c\") " pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:22.861067 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.861048    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk5h4\" (UniqueName: \"kubernetes.io/projected/424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c-kube-api-access-gk5h4\") pod \"test-curl-7651-140601040222224\" (UID: \"424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c\") " pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:22.964452 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:22.964434    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:23.095870 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:23.095845    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:23.097817 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:27:23.097792    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod424f3d58_5ea9_471e_bdcf_e6d2fc4dea7c.slice/crio-b2aa5a477c0a88035d191a08452704aff2ed71efefaff7951fee494be96a6245 WatchSource:0}: Error finding container b2aa5a477c0a88035d191a08452704aff2ed71efefaff7951fee494be96a6245: Status 404 returned error can't find the container with id b2aa5a477c0a88035d191a08452704aff2ed71efefaff7951fee494be96a6245
Jul 24 18:27:23.281931 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:23.281873    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601040222224" event={"ID":"424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c","Type":"ContainerStarted","Data":"b2aa5a477c0a88035d191a08452704aff2ed71efefaff7951fee494be96a6245"}
Jul 24 18:27:23.283139 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:23.283116    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6106a889f5162ffa4dd36dbaa4c7ebf0bc839a8b9de43f9cd165b0dad82d2d90"
Jul 24 18:27:23.283139 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:23.283137    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033885616"
Jul 24 18:27:23.293621 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:27:23.293598    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-140601033885616\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="65d6bab7-e06d-4834-bfd4-5fe069b02867" pod="opendatahub/test-curl-7651-140601033885616"
Jul 24 18:27:23.406967 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:27:23.406933    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-140601033885616\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="65d6bab7-e06d-4834-bfd4-5fe069b02867" pod="opendatahub/test-curl-7651-140601033885616"
Jul 24 18:27:23.409185 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:23.409161    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65d6bab7-e06d-4834-bfd4-5fe069b02867" path="/var/lib/kubelet/pods/65d6bab7-e06d-4834-bfd4-5fe069b02867/volumes"
Jul 24 18:27:24.290694 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:24.290659    2562 generic.go:356] "Generic (PLEG): container finished" podID="424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c" containerID="b878f60c1a50605345df394e89bd62d8308509dd6f490c57e8e5346dd0ac7780" exitCode=0
Jul 24 18:27:24.291123 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:24.290729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601040222224" event={"ID":"424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c","Type":"ContainerDied","Data":"b878f60c1a50605345df394e89bd62d8308509dd6f490c57e8e5346dd0ac7780"}
Jul 24 18:27:25.445571 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.445547    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:25.474500 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.474474    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7651-140601040222224_424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c/test-curl-7651-140601040222224/0.log"
Jul 24 18:27:25.502913 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.502891    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:25.508021 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.508000    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:25.564943 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.564864    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk5h4\" (UniqueName: \"kubernetes.io/projected/424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c-kube-api-access-gk5h4\") pod \"424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c\" (UID: \"424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c\") "
Jul 24 18:27:25.566786 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.566762    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c-kube-api-access-gk5h4" (OuterVolumeSpecName: "kube-api-access-gk5h4") pod "424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c" (UID: "424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c"). InnerVolumeSpecName "kube-api-access-gk5h4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:27:25.639491 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.639470    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:25.639905 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.639868    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c" containerName="test-curl-7651-140601040222224"
Jul 24 18:27:25.639905 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.639900    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c" containerName="test-curl-7651-140601040222224"
Jul 24 18:27:25.639995 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.639986    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c" containerName="test-curl-7651-140601040222224"
Jul 24 18:27:25.643078 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.643064    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:25.653614 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.653594    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:25.665786 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.665759    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk5h4\" (UniqueName: \"kubernetes.io/projected/424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c-kube-api-access-gk5h4\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:27:25.766411 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.766382    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh5jg\" (UniqueName: \"kubernetes.io/projected/bc8c88b0-cda2-4e40-9c0f-3430aba52cf2-kube-api-access-bh5jg\") pod \"test-curl-7651-140601040222224\" (UID: \"bc8c88b0-cda2-4e40-9c0f-3430aba52cf2\") " pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:25.867336 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.867278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh5jg\" (UniqueName: \"kubernetes.io/projected/bc8c88b0-cda2-4e40-9c0f-3430aba52cf2-kube-api-access-bh5jg\") pod \"test-curl-7651-140601040222224\" (UID: \"bc8c88b0-cda2-4e40-9c0f-3430aba52cf2\") " pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:25.876231 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.876207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh5jg\" (UniqueName: \"kubernetes.io/projected/bc8c88b0-cda2-4e40-9c0f-3430aba52cf2-kube-api-access-bh5jg\") pod \"test-curl-7651-140601040222224\" (UID: \"bc8c88b0-cda2-4e40-9c0f-3430aba52cf2\") " pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:25.953667 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:25.953645    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:26.096199 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:26.096172    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:26.097502 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:27:26.097478    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc8c88b0_cda2_4e40_9c0f_3430aba52cf2.slice/crio-a96345526c43ce71d6289927b1da852be4f55212403abadeeb60851cd528b935 WatchSource:0}: Error finding container a96345526c43ce71d6289927b1da852be4f55212403abadeeb60851cd528b935: Status 404 returned error can't find the container with id a96345526c43ce71d6289927b1da852be4f55212403abadeeb60851cd528b935
Jul 24 18:27:26.309374 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:26.309336    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601040222224" event={"ID":"bc8c88b0-cda2-4e40-9c0f-3430aba52cf2","Type":"ContainerStarted","Data":"a96345526c43ce71d6289927b1da852be4f55212403abadeeb60851cd528b935"}
Jul 24 18:27:26.310673 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:26.310656    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2aa5a477c0a88035d191a08452704aff2ed71efefaff7951fee494be96a6245"
Jul 24 18:27:26.310762 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:26.310681    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:26.322477 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:26.322450    2562 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7651-140601040222224" oldPodUID="424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c" podUID="bc8c88b0-cda2-4e40-9c0f-3430aba52cf2"
Jul 24 18:27:27.321509 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:27.321471    2562 generic.go:356] "Generic (PLEG): container finished" podID="bc8c88b0-cda2-4e40-9c0f-3430aba52cf2" containerID="1676603f20c1a3a5954d95b830a122d88417dd0f6fa1a524ae9c70ff7bfcff33" exitCode=0
Jul 24 18:27:27.322036 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:27.321556    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601040222224" event={"ID":"bc8c88b0-cda2-4e40-9c0f-3430aba52cf2","Type":"ContainerDied","Data":"1676603f20c1a3a5954d95b830a122d88417dd0f6fa1a524ae9c70ff7bfcff33"}
Jul 24 18:27:27.408103 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:27.408074    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c" path="/var/lib/kubelet/pods/424f3d58-5ea9-471e-bdcf-e6d2fc4dea7c/volumes"
Jul 24 18:27:28.469691 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.469667    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:28.490736 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.490713    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7651-140601040222224_bc8c88b0-cda2-4e40-9c0f-3430aba52cf2/test-curl-7651-140601040222224/0.log"
Jul 24 18:27:28.518992 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.518963    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:28.521352 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.521322    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-140601040222224"]
Jul 24 18:27:28.590118 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.590061    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh5jg\" (UniqueName: \"kubernetes.io/projected/bc8c88b0-cda2-4e40-9c0f-3430aba52cf2-kube-api-access-bh5jg\") pod \"bc8c88b0-cda2-4e40-9c0f-3430aba52cf2\" (UID: \"bc8c88b0-cda2-4e40-9c0f-3430aba52cf2\") "
Jul 24 18:27:28.591934 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.591911    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc8c88b0-cda2-4e40-9c0f-3430aba52cf2-kube-api-access-bh5jg" (OuterVolumeSpecName: "kube-api-access-bh5jg") pod "bc8c88b0-cda2-4e40-9c0f-3430aba52cf2" (UID: "bc8c88b0-cda2-4e40-9c0f-3430aba52cf2"). InnerVolumeSpecName "kube-api-access-bh5jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:27:28.643904 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.643873    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:28.644294 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.644281    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8c88b0-cda2-4e40-9c0f-3430aba52cf2" containerName="test-curl-7651-140601040222224"
Jul 24 18:27:28.644359 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.644295    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8c88b0-cda2-4e40-9c0f-3430aba52cf2" containerName="test-curl-7651-140601040222224"
Jul 24 18:27:28.644396 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.644371    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc8c88b0-cda2-4e40-9c0f-3430aba52cf2" containerName="test-curl-7651-140601040222224"
Jul 24 18:27:28.647342 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.647325    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:28.655231 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.654781    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:28.691217 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.691197    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz988\" (UniqueName: \"kubernetes.io/projected/32746f03-02e9-4ff4-be93-1d14a575c241-kube-api-access-kz988\") pod \"test-curl-7651-140601033754064\" (UID: \"32746f03-02e9-4ff4-be93-1d14a575c241\") " pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:28.691317 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.691300    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh5jg\" (UniqueName: \"kubernetes.io/projected/bc8c88b0-cda2-4e40-9c0f-3430aba52cf2-kube-api-access-bh5jg\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:27:28.792652 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.792627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz988\" (UniqueName: \"kubernetes.io/projected/32746f03-02e9-4ff4-be93-1d14a575c241-kube-api-access-kz988\") pod \"test-curl-7651-140601033754064\" (UID: \"32746f03-02e9-4ff4-be93-1d14a575c241\") " pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:28.800894 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.800863    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz988\" (UniqueName: \"kubernetes.io/projected/32746f03-02e9-4ff4-be93-1d14a575c241-kube-api-access-kz988\") pod \"test-curl-7651-140601033754064\" (UID: \"32746f03-02e9-4ff4-be93-1d14a575c241\") " pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:28.958055 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:28.957997    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:29.089495 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:29.089471    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:29.090500 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:27:29.090476    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32746f03_02e9_4ff4_be93_1d14a575c241.slice/crio-75ebe566050553ff4cbfac760d0b464358847fc1fedada8f4242ae19f62730c7 WatchSource:0}: Error finding container 75ebe566050553ff4cbfac760d0b464358847fc1fedada8f4242ae19f62730c7: Status 404 returned error can't find the container with id 75ebe566050553ff4cbfac760d0b464358847fc1fedada8f4242ae19f62730c7
Jul 24 18:27:29.334926 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:29.334900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601033754064" event={"ID":"32746f03-02e9-4ff4-be93-1d14a575c241","Type":"ContainerStarted","Data":"75ebe566050553ff4cbfac760d0b464358847fc1fedada8f4242ae19f62730c7"}
Jul 24 18:27:29.336117 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:29.336098    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:29.336117 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:29.336106    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a96345526c43ce71d6289927b1da852be4f55212403abadeeb60851cd528b935"
Jul 24 18:27:29.346455 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:27:29.346430    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-140601040222224\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="bc8c88b0-cda2-4e40-9c0f-3430aba52cf2" pod="opendatahub/test-curl-7651-140601040222224"
Jul 24 18:27:29.408170 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:29.408148    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc8c88b0-cda2-4e40-9c0f-3430aba52cf2" path="/var/lib/kubelet/pods/bc8c88b0-cda2-4e40-9c0f-3430aba52cf2/volumes"
Jul 24 18:27:30.344098 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:30.344066    2562 generic.go:356] "Generic (PLEG): container finished" podID="32746f03-02e9-4ff4-be93-1d14a575c241" containerID="125883f3f5fc3fd8fd1d558e7a87e3fb0c96a48b4205e0cf4dc987a47dbc7262" exitCode=0
Jul 24 18:27:30.344482 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:30.344212    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601033754064" event={"ID":"32746f03-02e9-4ff4-be93-1d14a575c241","Type":"ContainerDied","Data":"125883f3f5fc3fd8fd1d558e7a87e3fb0c96a48b4205e0cf4dc987a47dbc7262"}
Jul 24 18:27:31.479724 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.479699    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:31.517061 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.517038    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz988\" (UniqueName: \"kubernetes.io/projected/32746f03-02e9-4ff4-be93-1d14a575c241-kube-api-access-kz988\") pod \"32746f03-02e9-4ff4-be93-1d14a575c241\" (UID: \"32746f03-02e9-4ff4-be93-1d14a575c241\") "
Jul 24 18:27:31.519000 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.518973    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32746f03-02e9-4ff4-be93-1d14a575c241-kube-api-access-kz988" (OuterVolumeSpecName: "kube-api-access-kz988") pod "32746f03-02e9-4ff4-be93-1d14a575c241" (UID: "32746f03-02e9-4ff4-be93-1d14a575c241"). InnerVolumeSpecName "kube-api-access-kz988". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:27:31.531922 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.531900    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7651-140601033754064_32746f03-02e9-4ff4-be93-1d14a575c241/test-curl-7651-140601033754064/0.log"
Jul 24 18:27:31.569219 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.569195    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:31.575351 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.575331    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:31.618553 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.618504    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz988\" (UniqueName: \"kubernetes.io/projected/32746f03-02e9-4ff4-be93-1d14a575c241-kube-api-access-kz988\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:27:31.703912 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.703869    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:31.704379 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.704360    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32746f03-02e9-4ff4-be93-1d14a575c241" containerName="test-curl-7651-140601033754064"
Jul 24 18:27:31.704463 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.704382    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="32746f03-02e9-4ff4-be93-1d14a575c241" containerName="test-curl-7651-140601033754064"
Jul 24 18:27:31.704527 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.704488    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="32746f03-02e9-4ff4-be93-1d14a575c241" containerName="test-curl-7651-140601033754064"
Jul 24 18:27:31.707630 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.707612    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:31.714108 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.714090    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:31.820281 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.820256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h885v\" (UniqueName: \"kubernetes.io/projected/ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5-kube-api-access-h885v\") pod \"test-curl-7651-140601033754064\" (UID: \"ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5\") " pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:31.920974 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.920912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h885v\" (UniqueName: \"kubernetes.io/projected/ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5-kube-api-access-h885v\") pod \"test-curl-7651-140601033754064\" (UID: \"ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5\") " pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:31.929585 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:31.929566    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h885v\" (UniqueName: \"kubernetes.io/projected/ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5-kube-api-access-h885v\") pod \"test-curl-7651-140601033754064\" (UID: \"ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5\") " pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:32.018149 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:32.018128    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:32.147825 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:32.147795    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:32.148822 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:27:32.148800    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae59ced8_1ea8_49e3_82f9_eb1d6f71d6c5.slice/crio-eb702ce1ccfc85cd902eb52b97659acef84fc5e272f385ee32fa0299c6d61558 WatchSource:0}: Error finding container eb702ce1ccfc85cd902eb52b97659acef84fc5e272f385ee32fa0299c6d61558: Status 404 returned error can't find the container with id eb702ce1ccfc85cd902eb52b97659acef84fc5e272f385ee32fa0299c6d61558
Jul 24 18:27:32.356956 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:32.356930    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:32.357095 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:32.356931    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75ebe566050553ff4cbfac760d0b464358847fc1fedada8f4242ae19f62730c7"
Jul 24 18:27:32.358223 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:32.358198    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601033754064" event={"ID":"ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5","Type":"ContainerStarted","Data":"eb702ce1ccfc85cd902eb52b97659acef84fc5e272f385ee32fa0299c6d61558"}
Jul 24 18:27:32.368819 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:32.368798    2562 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7651-140601033754064" oldPodUID="32746f03-02e9-4ff4-be93-1d14a575c241" podUID="ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5"
Jul 24 18:27:33.365973 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:33.365938    2562 generic.go:356] "Generic (PLEG): container finished" podID="ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5" containerID="7c30e1e621f7f79b4bced7e953449f294afe2e16056b4892f002668b4c1d2605" exitCode=0
Jul 24 18:27:33.366375 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:33.366040    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-140601033754064" event={"ID":"ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5","Type":"ContainerDied","Data":"7c30e1e621f7f79b4bced7e953449f294afe2e16056b4892f002668b4c1d2605"}
Jul 24 18:27:33.407622 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:33.407595    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32746f03-02e9-4ff4-be93-1d14a575c241" path="/var/lib/kubelet/pods/32746f03-02e9-4ff4-be93-1d14a575c241/volumes"
Jul 24 18:27:34.511064 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:34.511043    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:34.535000 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:34.534972    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7651-140601033754064_ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5/test-curl-7651-140601033754064/0.log"
Jul 24 18:27:34.559528 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:34.559505    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:34.563273 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:34.563247    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-140601033754064"]
Jul 24 18:27:34.644668 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:34.644608    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h885v\" (UniqueName: \"kubernetes.io/projected/ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5-kube-api-access-h885v\") pod \"ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5\" (UID: \"ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5\") "
Jul 24 18:27:34.646662 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:34.646639    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5-kube-api-access-h885v" (OuterVolumeSpecName: "kube-api-access-h885v") pod "ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5" (UID: "ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5"). InnerVolumeSpecName "kube-api-access-h885v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:27:34.746032 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:34.746010    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h885v\" (UniqueName: \"kubernetes.io/projected/ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5-kube-api-access-h885v\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:27:35.379566 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:35.379520    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:35.379566 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:35.379552    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb702ce1ccfc85cd902eb52b97659acef84fc5e272f385ee32fa0299c6d61558"
Jul 24 18:27:35.389835 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:27:35.389809    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-140601033754064\" is forbidden: User \"system:node:ip-10-0-136-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-89.ec2.internal' and this object" podUID="ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5" pod="opendatahub/test-curl-7651-140601033754064"
Jul 24 18:27:35.407140 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:35.407117    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5" path="/var/lib/kubelet/pods/ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5/volumes"
Jul 24 18:27:39.991346 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:39.991314    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"]
Jul 24 18:27:39.991751 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:39.991737    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5" containerName="test-curl-7651-140601033754064"
Jul 24 18:27:39.991792 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:39.991754    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5" containerName="test-curl-7651-140601033754064"
Jul 24 18:27:39.991836 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:39.991826    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae59ced8-1ea8-49e3-82f9-eb1d6f71d6c5" containerName="test-curl-7651-140601033754064"
Jul 24 18:27:39.995122 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:39.995104    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:39.997972 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:39.997950    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-378627-serving-cert\""
Jul 24 18:27:40.006040 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.006021    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"]
Jul 24 18:27:40.088207 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.088183    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf5q2\" (UniqueName: \"kubernetes.io/projected/71057873-7e22-4597-a603-54d0ca0e2bfc-kube-api-access-mf5q2\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.088361 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.088291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71057873-7e22-4597-a603-54d0ca0e2bfc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.088361 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.088322    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71057873-7e22-4597-a603-54d0ca0e2bfc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.189387 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.189357    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71057873-7e22-4597-a603-54d0ca0e2bfc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.189536 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.189390    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71057873-7e22-4597-a603-54d0ca0e2bfc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.189536 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.189427    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5q2\" (UniqueName: \"kubernetes.io/projected/71057873-7e22-4597-a603-54d0ca0e2bfc-kube-api-access-mf5q2\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.189536 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:27:40.189531    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-serving-cert: secret "maas-api-e2e-ipp-a-378627-serving-cert" not found
Jul 24 18:27:40.189708 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:27:40.189596    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71057873-7e22-4597-a603-54d0ca0e2bfc-maas-api-tls podName:71057873-7e22-4597-a603-54d0ca0e2bfc nodeName:}" failed. No retries permitted until 2026-07-24 18:27:40.689577729 +0000 UTC m=+2867.894269508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71057873-7e22-4597-a603-54d0ca0e2bfc-maas-api-tls") pod "maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr" (UID: "71057873-7e22-4597-a603-54d0ca0e2bfc") : secret "maas-api-e2e-ipp-a-378627-serving-cert" not found
Jul 24 18:27:40.190136 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.190114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71057873-7e22-4597-a603-54d0ca0e2bfc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.198188 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.198170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf5q2\" (UniqueName: \"kubernetes.io/projected/71057873-7e22-4597-a603-54d0ca0e2bfc-kube-api-access-mf5q2\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.694451 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.694424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71057873-7e22-4597-a603-54d0ca0e2bfc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.696723 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.696694    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71057873-7e22-4597-a603-54d0ca0e2bfc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:40.907164 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:40.907134    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:41.255600 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:41.255571    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"]
Jul 24 18:27:41.256627 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:27:41.256601    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71057873_7e22_4597_a603_54d0ca0e2bfc.slice/crio-1f138b6c4b4c129d03394b9d25426f08212d55c5cfe4e47c166e6bb4846ba7e9 WatchSource:0}: Error finding container 1f138b6c4b4c129d03394b9d25426f08212d55c5cfe4e47c166e6bb4846ba7e9: Status 404 returned error can't find the container with id 1f138b6c4b4c129d03394b9d25426f08212d55c5cfe4e47c166e6bb4846ba7e9
Jul 24 18:27:41.416602 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:41.416560    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr" event={"ID":"71057873-7e22-4597-a603-54d0ca0e2bfc","Type":"ContainerStarted","Data":"1f138b6c4b4c129d03394b9d25426f08212d55c5cfe4e47c166e6bb4846ba7e9"}
Jul 24 18:27:42.424388 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:42.424354    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr" event={"ID":"71057873-7e22-4597-a603-54d0ca0e2bfc","Type":"ContainerStarted","Data":"537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691"}
Jul 24 18:27:42.424779 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:42.424437    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:42.451350 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:42.451298    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr" podStartSLOduration=3.003984199 podStartE2EDuration="3.451283258s" totalImagesPullingTime="447.299059ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:27:42.440451382 +0000 UTC m=+2869.645143299" watchObservedRunningTime="2026-07-24 18:27:42.451283258 +0000 UTC m=+2869.655975055"
Jul 24 18:27:48.435362 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:48.435332    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:27:54.394640 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.394610    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"]
Jul 24 18:27:54.400615 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.400592    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.403156 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.403134    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-cb47da-openshift-default-dockercfg-s9z6x\""
Jul 24 18:27:54.411133 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.411115    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"]
Jul 24 18:27:54.510662 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.510632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istiod-ca-cert\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.510858 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.510725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-certs\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.510956 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.510850    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-envoy\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.510956 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.510911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-data\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.510956 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.510939    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-credential-socket\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.511081 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.510971    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-podinfo\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.511081 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.511003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-token\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.511081 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.511057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-socket\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.511217 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.511084    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwngv\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-kube-api-access-bwngv\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.611861 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.611825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istiod-ca-cert\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612060 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.611908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-certs\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612060 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.611969    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-envoy\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612060 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.611992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-data\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612060 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-credential-socket\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612060 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612030    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-podinfo\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612060 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-token\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612365 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-socket\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612365 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwngv\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-kube-api-access-bwngv\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612472 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612401    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-certs\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612472 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612444    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-data\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612595 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612570    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-credential-socket\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612655 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istiod-ca-cert\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.612718 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.612702    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-socket\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.614294 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.614267    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-envoy\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.614622 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.614599    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-podinfo\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.623022 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.622999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-token\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.623184 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.623162    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwngv\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-kube-api-access-bwngv\") pod \"e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") " pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.712436 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.712360    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:54.852728 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.852699    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"]
Jul 24 18:27:54.852837 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:27:54.852750    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749b4c56_a3d4_40fd_a8cc_089901a6bfcb.slice/crio-50c61792f6a551133303190aabd2b0cef6ec2edcc10940e5b4d2a5a5077a31e5 WatchSource:0}: Error finding container 50c61792f6a551133303190aabd2b0cef6ec2edcc10940e5b4d2a5a5077a31e5: Status 404 returned error can't find the container with id 50c61792f6a551133303190aabd2b0cef6ec2edcc10940e5b4d2a5a5077a31e5
Jul 24 18:27:54.855023 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.854991    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 18:27:54.855104 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.855055    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 18:27:54.855104 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:54.855083    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 18:27:55.503979 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:55.503946    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" event={"ID":"749b4c56-a3d4-40fd-a8cc-089901a6bfcb","Type":"ContainerStarted","Data":"694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0"}
Jul 24 18:27:55.504454 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:55.504006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" event={"ID":"749b4c56-a3d4-40fd-a8cc-089901a6bfcb","Type":"ContainerStarted","Data":"50c61792f6a551133303190aabd2b0cef6ec2edcc10940e5b4d2a5a5077a31e5"}
Jul 24 18:27:55.530611 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:55.530568    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" podStartSLOduration=1.5305529820000001 podStartE2EDuration="1.530552982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:27:55.529808105 +0000 UTC m=+2882.734499903" watchObservedRunningTime="2026-07-24 18:27:55.530552982 +0000 UTC m=+2882.735244802"
Jul 24 18:27:55.712597 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:55.712569    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:55.717617 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:55.717585    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:56.509725 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:56.509692    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:56.510541 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:56.510525    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:27:59.048598 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.048570    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"]
Jul 24 18:27:59.052695 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.052674    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.055434 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.055411    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-cb47da-serving-cert\""
Jul 24 18:27:59.063659 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.063637    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"]
Jul 24 18:27:59.150244 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.150209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zszs\" (UniqueName: \"kubernetes.io/projected/6b070106-deec-4658-b65b-260a6356fe12-kube-api-access-2zszs\") pod \"maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.150411 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.150249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b070106-deec-4658-b65b-260a6356fe12-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.150411 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.150378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b070106-deec-4658-b65b-260a6356fe12-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.251298 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.251274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zszs\" (UniqueName: \"kubernetes.io/projected/6b070106-deec-4658-b65b-260a6356fe12-kube-api-access-2zszs\") pod \"maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.251480 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.251310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b070106-deec-4658-b65b-260a6356fe12-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.251480 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.251385    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b070106-deec-4658-b65b-260a6356fe12-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.252029 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.252006    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b070106-deec-4658-b65b-260a6356fe12-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.253742 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.253722    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b070106-deec-4658-b65b-260a6356fe12-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.259574 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.259546    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zszs\" (UniqueName: \"kubernetes.io/projected/6b070106-deec-4658-b65b-260a6356fe12-kube-api-access-2zszs\") pod \"maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.364647 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.364580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:27:59.508927 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.508675    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"]
Jul 24 18:27:59.513938 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:27:59.513908    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b070106_deec_4658_b65b_260a6356fe12.slice/crio-3bdee44c4d535f582eb6e0c456d2208bb5ef1c8b152869b85db2bdb461441c46 WatchSource:0}: Error finding container 3bdee44c4d535f582eb6e0c456d2208bb5ef1c8b152869b85db2bdb461441c46: Status 404 returned error can't find the container with id 3bdee44c4d535f582eb6e0c456d2208bb5ef1c8b152869b85db2bdb461441c46
Jul 24 18:27:59.528571 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:27:59.528545    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs" event={"ID":"6b070106-deec-4658-b65b-260a6356fe12","Type":"ContainerStarted","Data":"3bdee44c4d535f582eb6e0c456d2208bb5ef1c8b152869b85db2bdb461441c46"}
Jul 24 18:28:00.536474 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:00.536441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs" event={"ID":"6b070106-deec-4658-b65b-260a6356fe12","Type":"ContainerStarted","Data":"319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504"}
Jul 24 18:28:00.536873 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:00.536562    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:28:00.555428 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:00.555386    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs" podStartSLOduration=1.159667467 podStartE2EDuration="1.55537306s" totalImagesPullingTime="395.705593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:28:00.553390697 +0000 UTC m=+2887.758082497" watchObservedRunningTime="2026-07-24 18:28:00.55537306 +0000 UTC m=+2887.760064857"
Jul 24 18:28:06.547676 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:06.547648    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:28:15.629300 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:15.629264    2562 generic.go:356] "Generic (PLEG): container finished" podID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerID="694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0" exitCode=0
Jul 24 18:28:15.629735 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:15.629327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" event={"ID":"749b4c56-a3d4-40fd-a8cc-089901a6bfcb","Type":"ContainerDied","Data":"694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0"}
Jul 24 18:28:15.629735 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:15.629719    2562 scope.go:117] "RemoveContainer" containerID="694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0"
Jul 24 18:28:15.630871 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:15.630841    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 18:28:15.630962 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:15.630933    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 18:28:15.631000 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:15.630969    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 18:28:16.637445 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:16.637406    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" event={"ID":"749b4c56-a3d4-40fd-a8cc-089901a6bfcb","Type":"ContainerStarted","Data":"fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911"}
Jul 24 18:28:16.712702 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:16.712670    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:28:16.712944 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:16.712920    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Jul 24 18:28:16.713056 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:16.712978    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Jul 24 18:28:17.713327 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:17.713291    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Jul 24 18:28:17.713812 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:17.713349    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Jul 24 18:28:18.713631 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:18.713599    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Jul 24 18:28:18.714035 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:18.713648    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Jul 24 18:28:19.334806 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:19.334775    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w5rtf_1e1e7f00-128d-419c-8297-673e02231338/payload-processing/0.log"
Jul 24 18:28:19.712766 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:19.712677    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Jul 24 18:28:19.712766 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:19.712742    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Jul 24 18:28:20.716716 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:20.716682    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:28:20.717169 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:20.716972    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:28:20.717759 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:28:20.717739    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:29:06.563051 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:06.563010    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w5rtf_1e1e7f00-128d-419c-8297-673e02231338/payload-processing/0.log"
Jul 24 18:29:12.604143 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.604096    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"]
Jul 24 18:29:12.608957 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.608929    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:12.612376 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.612349    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-f94d2b-serving-cert\""
Jul 24 18:29:12.615609 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.615580    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"]
Jul 24 18:29:12.664774 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.664747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bl2z\" (UniqueName: \"kubernetes.io/projected/23bb5aab-74a8-4235-8a03-58a316db0b40-kube-api-access-6bl2z\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:12.664938 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.664813    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bb5aab-74a8-4235-8a03-58a316db0b40-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:12.664938 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.664858    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23bb5aab-74a8-4235-8a03-58a316db0b40-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:12.766123 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.766093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23bb5aab-74a8-4235-8a03-58a316db0b40-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:12.766266 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:29:12.766242    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-serving-cert: secret "maas-api-e2e-ipp-cleanup-f94d2b-serving-cert" not found
Jul 24 18:29:12.766266 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.766251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bl2z\" (UniqueName: \"kubernetes.io/projected/23bb5aab-74a8-4235-8a03-58a316db0b40-kube-api-access-6bl2z\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:12.766378 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:29:12.766302    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23bb5aab-74a8-4235-8a03-58a316db0b40-maas-api-tls podName:23bb5aab-74a8-4235-8a03-58a316db0b40 nodeName:}" failed. No retries permitted until 2026-07-24 18:29:13.266287314 +0000 UTC m=+2960.470979093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23bb5aab-74a8-4235-8a03-58a316db0b40-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j" (UID: "23bb5aab-74a8-4235-8a03-58a316db0b40") : secret "maas-api-e2e-ipp-cleanup-f94d2b-serving-cert" not found
Jul 24 18:29:12.766441 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.766384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bb5aab-74a8-4235-8a03-58a316db0b40-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:12.766950 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.766933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bb5aab-74a8-4235-8a03-58a316db0b40-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:12.775083 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:12.775065    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bl2z\" (UniqueName: \"kubernetes.io/projected/23bb5aab-74a8-4235-8a03-58a316db0b40-kube-api-access-6bl2z\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:13.207456 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.207424    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"]
Jul 24 18:29:13.211110 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.211094    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:13.213755 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.213734    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-f94d2b-dockercfg-rjksf\""
Jul 24 18:29:13.214016 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.213998    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-f94d2b\""
Jul 24 18:29:13.220756 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.220731    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"]
Jul 24 18:29:13.241674 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.241651    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"]
Jul 24 18:29:13.245162 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.245143    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:13.257756 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.257731    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"]
Jul 24 18:29:13.271088 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.271060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp5vn\" (UniqueName: \"kubernetes.io/projected/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-kube-api-access-mp5vn\") pod \"payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26\" (UID: \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:13.271232 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.271134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18feecc5-0129-48a5-b40a-69c404b2f284-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9\" (UID: \"18feecc5-0129-48a5-b40a-69c404b2f284\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:13.271232 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.271185    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fk5w\" (UniqueName: \"kubernetes.io/projected/18feecc5-0129-48a5-b40a-69c404b2f284-kube-api-access-9fk5w\") pod \"payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9\" (UID: \"18feecc5-0129-48a5-b40a-69c404b2f284\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:13.271362 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.271236    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23bb5aab-74a8-4235-8a03-58a316db0b40-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:13.271362 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.271281    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26\" (UID: \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:13.273713 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.273683    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23bb5aab-74a8-4235-8a03-58a316db0b40-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:13.372762 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.372725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26\" (UID: \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:13.372964 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.372806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp5vn\" (UniqueName: \"kubernetes.io/projected/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-kube-api-access-mp5vn\") pod \"payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26\" (UID: \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:13.372964 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.372837    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18feecc5-0129-48a5-b40a-69c404b2f284-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9\" (UID: \"18feecc5-0129-48a5-b40a-69c404b2f284\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:13.372964 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.372867    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fk5w\" (UniqueName: \"kubernetes.io/projected/18feecc5-0129-48a5-b40a-69c404b2f284-kube-api-access-9fk5w\") pod \"payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9\" (UID: \"18feecc5-0129-48a5-b40a-69c404b2f284\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:13.373704 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.373681    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26\" (UID: \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:13.373771 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.373681    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18feecc5-0129-48a5-b40a-69c404b2f284-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9\" (UID: \"18feecc5-0129-48a5-b40a-69c404b2f284\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:13.381631 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.381609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp5vn\" (UniqueName: \"kubernetes.io/projected/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-kube-api-access-mp5vn\") pod \"payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26\" (UID: \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:13.382055 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.382023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fk5w\" (UniqueName: \"kubernetes.io/projected/18feecc5-0129-48a5-b40a-69c404b2f284-kube-api-access-9fk5w\") pod \"payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9\" (UID: \"18feecc5-0129-48a5-b40a-69c404b2f284\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:13.523030 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.522971    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:13.524106 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.524080    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:13.557186 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.556771    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:13.695056 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.695019    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"]
Jul 24 18:29:13.696452 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:29:13.696424    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23bb5aab_74a8_4235_8a03_58a316db0b40.slice/crio-78778a60ccb17e917323775d4a8bbb751a799b9993e4af6de6646478f212aa72 WatchSource:0}: Error finding container 78778a60ccb17e917323775d4a8bbb751a799b9993e4af6de6646478f212aa72: Status 404 returned error can't find the container with id 78778a60ccb17e917323775d4a8bbb751a799b9993e4af6de6646478f212aa72
Jul 24 18:29:13.934940 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.934853    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"]
Jul 24 18:29:13.935323 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:29:13.935281    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cd8a27_7f3c_49dd_8eee_5895ed84faa6.slice/crio-79ad9cd4efc6a4b224b937153e8430defbbe55fcedc943124a78336bee681a32 WatchSource:0}: Error finding container 79ad9cd4efc6a4b224b937153e8430defbbe55fcedc943124a78336bee681a32: Status 404 returned error can't find the container with id 79ad9cd4efc6a4b224b937153e8430defbbe55fcedc943124a78336bee681a32
Jul 24 18:29:13.962624 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:29:13.962602    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18feecc5_0129_48a5_b40a_69c404b2f284.slice/crio-7c30e6ebdbb42e9b58747913e1b38b76108446bfdee178643055e564ec3e19d9 WatchSource:0}: Error finding container 7c30e6ebdbb42e9b58747913e1b38b76108446bfdee178643055e564ec3e19d9: Status 404 returned error can't find the container with id 7c30e6ebdbb42e9b58747913e1b38b76108446bfdee178643055e564ec3e19d9
Jul 24 18:29:13.963439 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:13.963417    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"]
Jul 24 18:29:14.007328 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:14.007306    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j" event={"ID":"23bb5aab-74a8-4235-8a03-58a316db0b40","Type":"ContainerStarted","Data":"78778a60ccb17e917323775d4a8bbb751a799b9993e4af6de6646478f212aa72"}
Jul 24 18:29:14.008736 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:14.008712    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9" event={"ID":"18feecc5-0129-48a5-b40a-69c404b2f284","Type":"ContainerStarted","Data":"7c30e6ebdbb42e9b58747913e1b38b76108446bfdee178643055e564ec3e19d9"}
Jul 24 18:29:14.010316 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:14.010291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26" event={"ID":"39cd8a27-7f3c-49dd-8eee-5895ed84faa6","Type":"ContainerStarted","Data":"79ad9cd4efc6a4b224b937153e8430defbbe55fcedc943124a78336bee681a32"}
Jul 24 18:29:15.017754 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:15.017719    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j" event={"ID":"23bb5aab-74a8-4235-8a03-58a316db0b40","Type":"ContainerStarted","Data":"979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc"}
Jul 24 18:29:15.018202 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:15.017782    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:15.019327 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:15.019303    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9" event={"ID":"18feecc5-0129-48a5-b40a-69c404b2f284","Type":"ContainerStarted","Data":"8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4"}
Jul 24 18:29:15.019463 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:15.019362    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:15.020758 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:15.020732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26" event={"ID":"39cd8a27-7f3c-49dd-8eee-5895ed84faa6","Type":"ContainerStarted","Data":"cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383"}
Jul 24 18:29:15.020837 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:15.020799    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:15.038220 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:15.038182    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j" podStartSLOduration=2.6666980430000002 podStartE2EDuration="3.038169086s" totalImagesPullingTime="371.471043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:29:15.034480832 +0000 UTC m=+2962.239172641" watchObservedRunningTime="2026-07-24 18:29:15.038169086 +0000 UTC m=+2962.242860882"
Jul 24 18:29:15.053250 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:15.053210    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26" podStartSLOduration=2.05319841 podStartE2EDuration="2.05319841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:29:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:29:15.051128916 +0000 UTC m=+2962.255820715" watchObservedRunningTime="2026-07-24 18:29:15.05319841 +0000 UTC m=+2962.257890228"
Jul 24 18:29:15.067182 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:15.067148    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9" podStartSLOduration=2.067137481 podStartE2EDuration="2.067137481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:29:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:29:15.066666401 +0000 UTC m=+2962.271358200" watchObservedRunningTime="2026-07-24 18:29:15.067137481 +0000 UTC m=+2962.271829278"
Jul 24 18:29:21.034472 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:21.034445    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:26.030932 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:26.030873    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:26.031387 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:26.031110    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:43.401550 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.401089    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"]
Jul 24 18:29:43.407532 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.407493    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j" podUID="23bb5aab-74a8-4235-8a03-58a316db0b40" containerName="maas-api" containerID="cri-o://979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc" gracePeriod=30
Jul 24 18:29:43.495055 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.495028    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"]
Jul 24 18:29:43.495292 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.495248    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9" podUID="18feecc5-0129-48a5-b40a-69c404b2f284" containerName="payload-processing" containerID="cri-o://8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4" gracePeriod=30
Jul 24 18:29:43.501913 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.501869    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"]
Jul 24 18:29:43.502121 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.502098    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26" podUID="39cd8a27-7f3c-49dd-8eee-5895ed84faa6" containerName="payload-pre-processing" containerID="cri-o://cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383" gracePeriod=30
Jul 24 18:29:43.764915 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.764872    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:43.813807 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.813783    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:43.814295 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.814271    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bl2z\" (UniqueName: \"kubernetes.io/projected/23bb5aab-74a8-4235-8a03-58a316db0b40-kube-api-access-6bl2z\") pod \"23bb5aab-74a8-4235-8a03-58a316db0b40\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") "
Jul 24 18:29:43.814405 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.814309    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23bb5aab-74a8-4235-8a03-58a316db0b40-maas-api-tls\") pod \"23bb5aab-74a8-4235-8a03-58a316db0b40\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") "
Jul 24 18:29:43.814470 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.814427    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bb5aab-74a8-4235-8a03-58a316db0b40-openshift-service-ca\") pod \"23bb5aab-74a8-4235-8a03-58a316db0b40\" (UID: \"23bb5aab-74a8-4235-8a03-58a316db0b40\") "
Jul 24 18:29:43.814802 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.814777    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23bb5aab-74a8-4235-8a03-58a316db0b40-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "23bb5aab-74a8-4235-8a03-58a316db0b40" (UID: "23bb5aab-74a8-4235-8a03-58a316db0b40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:29:43.816507 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.816487    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23bb5aab-74a8-4235-8a03-58a316db0b40-kube-api-access-6bl2z" (OuterVolumeSpecName: "kube-api-access-6bl2z") pod "23bb5aab-74a8-4235-8a03-58a316db0b40" (UID: "23bb5aab-74a8-4235-8a03-58a316db0b40"). InnerVolumeSpecName "kube-api-access-6bl2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:29:43.816593 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.816516    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23bb5aab-74a8-4235-8a03-58a316db0b40-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "23bb5aab-74a8-4235-8a03-58a316db0b40" (UID: "23bb5aab-74a8-4235-8a03-58a316db0b40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:29:43.817437 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.817419    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:43.915051 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.914993    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18feecc5-0129-48a5-b40a-69c404b2f284-plugins-config-volume\") pod \"18feecc5-0129-48a5-b40a-69c404b2f284\" (UID: \"18feecc5-0129-48a5-b40a-69c404b2f284\") "
Jul 24 18:29:43.915051 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.915038    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp5vn\" (UniqueName: \"kubernetes.io/projected/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-kube-api-access-mp5vn\") pod \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\" (UID: \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\") "
Jul 24 18:29:43.915214 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.915094    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-plugins-config-volume\") pod \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\" (UID: \"39cd8a27-7f3c-49dd-8eee-5895ed84faa6\") "
Jul 24 18:29:43.915214 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.915154    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fk5w\" (UniqueName: \"kubernetes.io/projected/18feecc5-0129-48a5-b40a-69c404b2f284-kube-api-access-9fk5w\") pod \"18feecc5-0129-48a5-b40a-69c404b2f284\" (UID: \"18feecc5-0129-48a5-b40a-69c404b2f284\") "
Jul 24 18:29:43.915416 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.915387    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18feecc5-0129-48a5-b40a-69c404b2f284-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18feecc5-0129-48a5-b40a-69c404b2f284" (UID: "18feecc5-0129-48a5-b40a-69c404b2f284"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:29:43.915535 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.915511    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39cd8a27-7f3c-49dd-8eee-5895ed84faa6" (UID: "39cd8a27-7f3c-49dd-8eee-5895ed84faa6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:29:43.915605 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.915525    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18feecc5-0129-48a5-b40a-69c404b2f284-plugins-config-volume\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:29:43.915605 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.915563    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bb5aab-74a8-4235-8a03-58a316db0b40-openshift-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:29:43.915605 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.915582    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bl2z\" (UniqueName: \"kubernetes.io/projected/23bb5aab-74a8-4235-8a03-58a316db0b40-kube-api-access-6bl2z\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:29:43.915605 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.915598    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23bb5aab-74a8-4235-8a03-58a316db0b40-maas-api-tls\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:29:43.917118 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.917099    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18feecc5-0129-48a5-b40a-69c404b2f284-kube-api-access-9fk5w" (OuterVolumeSpecName: "kube-api-access-9fk5w") pod "18feecc5-0129-48a5-b40a-69c404b2f284" (UID: "18feecc5-0129-48a5-b40a-69c404b2f284"). InnerVolumeSpecName "kube-api-access-9fk5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:29:43.917324 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:43.917303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-kube-api-access-mp5vn" (OuterVolumeSpecName: "kube-api-access-mp5vn") pod "39cd8a27-7f3c-49dd-8eee-5895ed84faa6" (UID: "39cd8a27-7f3c-49dd-8eee-5895ed84faa6"). InnerVolumeSpecName "kube-api-access-mp5vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:29:44.016780 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.016755    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-plugins-config-volume\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:29:44.016780 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.016776    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fk5w\" (UniqueName: \"kubernetes.io/projected/18feecc5-0129-48a5-b40a-69c404b2f284-kube-api-access-9fk5w\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:29:44.016947 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.016787    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp5vn\" (UniqueName: \"kubernetes.io/projected/39cd8a27-7f3c-49dd-8eee-5895ed84faa6-kube-api-access-mp5vn\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:29:44.213190 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.213120    2562 generic.go:356] "Generic (PLEG): container finished" podID="23bb5aab-74a8-4235-8a03-58a316db0b40" containerID="979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc" exitCode=0
Jul 24 18:29:44.213190 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.213159    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"
Jul 24 18:29:44.213354 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.213209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j" event={"ID":"23bb5aab-74a8-4235-8a03-58a316db0b40","Type":"ContainerDied","Data":"979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc"}
Jul 24 18:29:44.213354 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.213268    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j" event={"ID":"23bb5aab-74a8-4235-8a03-58a316db0b40","Type":"ContainerDied","Data":"78778a60ccb17e917323775d4a8bbb751a799b9993e4af6de6646478f212aa72"}
Jul 24 18:29:44.213354 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.213301    2562 scope.go:117] "RemoveContainer" containerID="979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc"
Jul 24 18:29:44.214700 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.214677    2562 generic.go:356] "Generic (PLEG): container finished" podID="18feecc5-0129-48a5-b40a-69c404b2f284" containerID="8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4" exitCode=0
Jul 24 18:29:44.214817 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.214705    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9" event={"ID":"18feecc5-0129-48a5-b40a-69c404b2f284","Type":"ContainerDied","Data":"8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4"}
Jul 24 18:29:44.214817 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.214727    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"
Jul 24 18:29:44.214817 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.214752    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9" event={"ID":"18feecc5-0129-48a5-b40a-69c404b2f284","Type":"ContainerDied","Data":"7c30e6ebdbb42e9b58747913e1b38b76108446bfdee178643055e564ec3e19d9"}
Jul 24 18:29:44.216260 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.216239    2562 generic.go:356] "Generic (PLEG): container finished" podID="39cd8a27-7f3c-49dd-8eee-5895ed84faa6" containerID="cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383" exitCode=0
Jul 24 18:29:44.216369 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.216260    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26" event={"ID":"39cd8a27-7f3c-49dd-8eee-5895ed84faa6","Type":"ContainerDied","Data":"cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383"}
Jul 24 18:29:44.216369 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.216289    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"
Jul 24 18:29:44.216369 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.216294    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26" event={"ID":"39cd8a27-7f3c-49dd-8eee-5895ed84faa6","Type":"ContainerDied","Data":"79ad9cd4efc6a4b224b937153e8430defbbe55fcedc943124a78336bee681a32"}
Jul 24 18:29:44.234816 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.234800    2562 scope.go:117] "RemoveContainer" containerID="979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc"
Jul 24 18:29:44.235175 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:29:44.235146    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc\": container with ID starting with 979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc not found: ID does not exist" containerID="979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc"
Jul 24 18:29:44.235254 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.235179    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc"} err="failed to get container status \"979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc\": rpc error: code = NotFound desc = could not find container \"979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc\": container with ID starting with 979395e92a5f5ccfe78c94eb4e40211a61718a70025f28dd8b6ee52bd06c47cc not found: ID does not exist"
Jul 24 18:29:44.235254 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.235194    2562 scope.go:117] "RemoveContainer" containerID="8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4"
Jul 24 18:29:44.243707 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.243687    2562 scope.go:117] "RemoveContainer" containerID="8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4"
Jul 24 18:29:44.243931 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:29:44.243912    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4\": container with ID starting with 8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4 not found: ID does not exist" containerID="8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4"
Jul 24 18:29:44.244004 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.243936    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4"} err="failed to get container status \"8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4\": rpc error: code = NotFound desc = could not find container \"8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4\": container with ID starting with 8f3d078bf7d189eb179d39f6a83d14afac0db6eb0fdadd747fcd0a503f1b4ac4 not found: ID does not exist"
Jul 24 18:29:44.244004 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.243951    2562 scope.go:117] "RemoveContainer" containerID="cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383"
Jul 24 18:29:44.250719 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.250693    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"]
Jul 24 18:29:44.253613 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.253516    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f94d2b-5798b9766b-pmcm9"]
Jul 24 18:29:44.255707 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.255685    2562 scope.go:117] "RemoveContainer" containerID="cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383"
Jul 24 18:29:44.256849 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:29:44.256827    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383\": container with ID starting with cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383 not found: ID does not exist" containerID="cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383"
Jul 24 18:29:44.256955 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.256857    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383"} err="failed to get container status \"cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383\": rpc error: code = NotFound desc = could not find container \"cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383\": container with ID starting with cde100021efb17356f6e1391025016d82be8a9a8b5e45d037f9e522b9fdab383 not found: ID does not exist"
Jul 24 18:29:44.266524 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.266503    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"]
Jul 24 18:29:44.270738 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.270715    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f94d2b-85f9db9f74-75h2j"]
Jul 24 18:29:44.281464 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.281440    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"]
Jul 24 18:29:44.286738 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:44.286719    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f94d2b-68cf6d6456-p6j26"]
Jul 24 18:29:45.408496 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:45.408463    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18feecc5-0129-48a5-b40a-69c404b2f284" path="/var/lib/kubelet/pods/18feecc5-0129-48a5-b40a-69c404b2f284/volumes"
Jul 24 18:29:45.408917 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:45.408899    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23bb5aab-74a8-4235-8a03-58a316db0b40" path="/var/lib/kubelet/pods/23bb5aab-74a8-4235-8a03-58a316db0b40/volumes"
Jul 24 18:29:45.409268 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:45.409255    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39cd8a27-7f3c-49dd-8eee-5895ed84faa6" path="/var/lib/kubelet/pods/39cd8a27-7f3c-49dd-8eee-5895ed84faa6/volumes"
Jul 24 18:29:53.577566 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:53.577540    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:29:53.585818 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:53.585796    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:29:53.592998 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:53.592981    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:29:53.600694 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:53.600678    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r59tv_edf57894-9068-4294-952a-76003b90530f/ovn-acl-logging/0.log"
Jul 24 18:29:56.101695 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.101652    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"]
Jul 24 18:29:56.102348 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.102322    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cd8a27-7f3c-49dd-8eee-5895ed84faa6" containerName="payload-pre-processing"
Jul 24 18:29:56.102440 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.102350    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cd8a27-7f3c-49dd-8eee-5895ed84faa6" containerName="payload-pre-processing"
Jul 24 18:29:56.102440 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.102363    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23bb5aab-74a8-4235-8a03-58a316db0b40" containerName="maas-api"
Jul 24 18:29:56.102440 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.102372    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="23bb5aab-74a8-4235-8a03-58a316db0b40" containerName="maas-api"
Jul 24 18:29:56.102440 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.102400    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18feecc5-0129-48a5-b40a-69c404b2f284" containerName="payload-processing"
Jul 24 18:29:56.102440 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.102410    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="18feecc5-0129-48a5-b40a-69c404b2f284" containerName="payload-processing"
Jul 24 18:29:56.103143 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.102665    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cd8a27-7f3c-49dd-8eee-5895ed84faa6" containerName="payload-pre-processing"
Jul 24 18:29:56.103143 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.102686    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="23bb5aab-74a8-4235-8a03-58a316db0b40" containerName="maas-api"
Jul 24 18:29:56.103143 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.102699    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="18feecc5-0129-48a5-b40a-69c404b2f284" containerName="payload-processing"
Jul 24 18:29:56.107917 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.107871    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"
Jul 24 18:29:56.111118 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.111026    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:29:56.113633 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.113605    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"]
Jul 24 18:29:56.228822 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.228800    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b948496e-bdd0-4313-9e18-34f75ba5efca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr\" (UID: \"b948496e-bdd0-4313-9e18-34f75ba5efca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"
Jul 24 18:29:56.330032 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.330005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b948496e-bdd0-4313-9e18-34f75ba5efca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr\" (UID: \"b948496e-bdd0-4313-9e18-34f75ba5efca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"
Jul 24 18:29:56.330682 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.330655    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b948496e-bdd0-4313-9e18-34f75ba5efca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr\" (UID: \"b948496e-bdd0-4313-9e18-34f75ba5efca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"
Jul 24 18:29:56.422012 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.421956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"
Jul 24 18:29:56.559063 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:56.559036    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"]
Jul 24 18:29:56.560244 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:29:56.560216    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb948496e_bdd0_4313_9e18_34f75ba5efca.slice/crio-009ae645eba629c2c794f50cbde0dcc7324046c79bead4d58282baef1dbcd911 WatchSource:0}: Error finding container 009ae645eba629c2c794f50cbde0dcc7324046c79bead4d58282baef1dbcd911: Status 404 returned error can't find the container with id 009ae645eba629c2c794f50cbde0dcc7324046c79bead4d58282baef1dbcd911
Jul 24 18:29:57.302333 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:57.302306    2562 generic.go:356] "Generic (PLEG): container finished" podID="b948496e-bdd0-4313-9e18-34f75ba5efca" containerID="e4744d0e3183e1115155171b57a0bdbd50168e118fffcb4885ad73c275ab7050" exitCode=0
Jul 24 18:29:57.302638 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:57.302389    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr" event={"ID":"b948496e-bdd0-4313-9e18-34f75ba5efca","Type":"ContainerDied","Data":"e4744d0e3183e1115155171b57a0bdbd50168e118fffcb4885ad73c275ab7050"}
Jul 24 18:29:57.302638 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:57.302441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr" event={"ID":"b948496e-bdd0-4313-9e18-34f75ba5efca","Type":"ContainerStarted","Data":"009ae645eba629c2c794f50cbde0dcc7324046c79bead4d58282baef1dbcd911"}
Jul 24 18:29:58.447657 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:58.447636    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"
Jul 24 18:29:58.550816 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:58.550786    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b948496e-bdd0-4313-9e18-34f75ba5efca-maas-api-service-ca\") pod \"b948496e-bdd0-4313-9e18-34f75ba5efca\" (UID: \"b948496e-bdd0-4313-9e18-34f75ba5efca\") "
Jul 24 18:29:58.551200 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:58.551177    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b948496e-bdd0-4313-9e18-34f75ba5efca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b948496e-bdd0-4313-9e18-34f75ba5efca" (UID: "b948496e-bdd0-4313-9e18-34f75ba5efca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:29:58.651684 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:58.651623    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b948496e-bdd0-4313-9e18-34f75ba5efca-maas-api-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:29:59.316684 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:59.316652    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"
Jul 24 18:29:59.316837 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:59.316703    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr" event={"ID":"b948496e-bdd0-4313-9e18-34f75ba5efca","Type":"ContainerDied","Data":"009ae645eba629c2c794f50cbde0dcc7324046c79bead4d58282baef1dbcd911"}
Jul 24 18:29:59.316837 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:29:59.316769    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="009ae645eba629c2c794f50cbde0dcc7324046c79bead4d58282baef1dbcd911"
Jul 24 18:30:00.142397 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.142364    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb"]
Jul 24 18:30:00.142893 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.142858    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b948496e-bdd0-4313-9e18-34f75ba5efca" containerName="revoke-keys"
Jul 24 18:30:00.142893 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.142894    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b948496e-bdd0-4313-9e18-34f75ba5efca" containerName="revoke-keys"
Jul 24 18:30:00.143022 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.143009    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b948496e-bdd0-4313-9e18-34f75ba5efca" containerName="revoke-keys"
Jul 24 18:30:00.148143 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.148123    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6"]
Jul 24 18:30:00.148265 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.148232    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb"
Jul 24 18:30:00.150806 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.150788    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:30:00.151641 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.151626    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6"
Jul 24 18:30:00.159276 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.159230    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb"]
Jul 24 18:30:00.162334 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.162310    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6"]
Jul 24 18:30:00.163047 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.163025    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb"
Jul 24 18:30:00.171565 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.171548    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6"
Jul 24 18:30:00.315525 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.315492    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb"]
Jul 24 18:30:00.315974 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:30:00.315927    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e3eec7_1d73_4d36_8720_b200d6c1e232.slice/crio-959281b662c6437e0f329553e0623b2073c823be298d40342faa80dd02b6829e WatchSource:0}: Error finding container 959281b662c6437e0f329553e0623b2073c823be298d40342faa80dd02b6829e: Status 404 returned error can't find the container with id 959281b662c6437e0f329553e0623b2073c823be298d40342faa80dd02b6829e
Jul 24 18:30:00.324694 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.324669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb" event={"ID":"14e3eec7-1d73-4d36-8720-b200d6c1e232","Type":"ContainerStarted","Data":"959281b662c6437e0f329553e0623b2073c823be298d40342faa80dd02b6829e"}
Jul 24 18:30:00.546504 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:00.546477    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6"]
Jul 24 18:30:00.548209 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:30:00.548185    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d1bb7e_88ac_482d_b3c7_9b7d3d267a84.slice/crio-c3e769a04aa5b85fe9e169695fb26000b185193ab6fe59728b575733db289ba2 WatchSource:0}: Error finding container c3e769a04aa5b85fe9e169695fb26000b185193ab6fe59728b575733db289ba2: Status 404 returned error can't find the container with id c3e769a04aa5b85fe9e169695fb26000b185193ab6fe59728b575733db289ba2
Jul 24 18:30:01.333564 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:01.333530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6" event={"ID":"77d1bb7e-88ac-482d-b3c7-9b7d3d267a84","Type":"ContainerStarted","Data":"b613f1f1d6902a29e4e06ca4238c2db34312765ddde617c5eb4e32e7aea6703e"}
Jul 24 18:30:01.334005 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:01.333594    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6" event={"ID":"77d1bb7e-88ac-482d-b3c7-9b7d3d267a84","Type":"ContainerStarted","Data":"c3e769a04aa5b85fe9e169695fb26000b185193ab6fe59728b575733db289ba2"}
Jul 24 18:30:01.335408 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:01.335382    2562 generic.go:356] "Generic (PLEG): container finished" podID="14e3eec7-1d73-4d36-8720-b200d6c1e232" containerID="60ee4f680aa9793bfe04fe94bc6932e5608a8d92eb020eaaff95dd3de7c932d3" exitCode=0
Jul 24 18:30:01.335532 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:01.335469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb" event={"ID":"14e3eec7-1d73-4d36-8720-b200d6c1e232","Type":"ContainerDied","Data":"60ee4f680aa9793bfe04fe94bc6932e5608a8d92eb020eaaff95dd3de7c932d3"}
Jul 24 18:30:01.355818 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:01.355769    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6" podStartSLOduration=1.355754756 podStartE2EDuration="1.355754756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:30:01.353288608 +0000 UTC m=+3008.557980409" watchObservedRunningTime="2026-07-24 18:30:01.355754756 +0000 UTC m=+3008.560446579"
Jul 24 18:30:02.343661 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:02.343627    2562 generic.go:356] "Generic (PLEG): container finished" podID="77d1bb7e-88ac-482d-b3c7-9b7d3d267a84" containerID="b613f1f1d6902a29e4e06ca4238c2db34312765ddde617c5eb4e32e7aea6703e" exitCode=0
Jul 24 18:30:02.344125 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:02.343705    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6" event={"ID":"77d1bb7e-88ac-482d-b3c7-9b7d3d267a84","Type":"ContainerDied","Data":"b613f1f1d6902a29e4e06ca4238c2db34312765ddde617c5eb4e32e7aea6703e"}
Jul 24 18:30:02.482029 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:02.482006    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb"
Jul 24 18:30:03.363630 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:03.363594    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb"
Jul 24 18:30:03.364138 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:03.363628    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748630-5d8nb" event={"ID":"14e3eec7-1d73-4d36-8720-b200d6c1e232","Type":"ContainerDied","Data":"959281b662c6437e0f329553e0623b2073c823be298d40342faa80dd02b6829e"}
Jul 24 18:30:03.364138 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:03.363707    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="959281b662c6437e0f329553e0623b2073c823be298d40342faa80dd02b6829e"
Jul 24 18:30:03.521289 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:03.521266    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6"
Jul 24 18:30:04.370660 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:04.370627    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6"
Jul 24 18:30:04.370660 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:04.370642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6" event={"ID":"77d1bb7e-88ac-482d-b3c7-9b7d3d267a84","Type":"ContainerDied","Data":"c3e769a04aa5b85fe9e169695fb26000b185193ab6fe59728b575733db289ba2"}
Jul 24 18:30:04.371097 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:04.370685    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3e769a04aa5b85fe9e169695fb26000b185193ab6fe59728b575733db289ba2"
Jul 24 18:30:14.927456 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:14.927420    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"]
Jul 24 18:30:14.928001 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:14.927630    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr" podUID="71057873-7e22-4597-a603-54d0ca0e2bfc" containerName="maas-api" containerID="cri-o://537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691" gracePeriod=30
Jul 24 18:30:15.196626 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.196602    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:30:15.290558 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.290533    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf5q2\" (UniqueName: \"kubernetes.io/projected/71057873-7e22-4597-a603-54d0ca0e2bfc-kube-api-access-mf5q2\") pod \"71057873-7e22-4597-a603-54d0ca0e2bfc\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") "
Jul 24 18:30:15.290699 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.290581    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71057873-7e22-4597-a603-54d0ca0e2bfc-maas-api-tls\") pod \"71057873-7e22-4597-a603-54d0ca0e2bfc\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") "
Jul 24 18:30:15.290699 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.290652    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71057873-7e22-4597-a603-54d0ca0e2bfc-openshift-service-ca\") pod \"71057873-7e22-4597-a603-54d0ca0e2bfc\" (UID: \"71057873-7e22-4597-a603-54d0ca0e2bfc\") "
Jul 24 18:30:15.291056 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.291032    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71057873-7e22-4597-a603-54d0ca0e2bfc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71057873-7e22-4597-a603-54d0ca0e2bfc" (UID: "71057873-7e22-4597-a603-54d0ca0e2bfc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:30:15.292548 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.292499    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71057873-7e22-4597-a603-54d0ca0e2bfc-kube-api-access-mf5q2" (OuterVolumeSpecName: "kube-api-access-mf5q2") pod "71057873-7e22-4597-a603-54d0ca0e2bfc" (UID: "71057873-7e22-4597-a603-54d0ca0e2bfc"). InnerVolumeSpecName "kube-api-access-mf5q2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:30:15.292648 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.292592    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71057873-7e22-4597-a603-54d0ca0e2bfc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71057873-7e22-4597-a603-54d0ca0e2bfc" (UID: "71057873-7e22-4597-a603-54d0ca0e2bfc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:30:15.391640 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.391616    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf5q2\" (UniqueName: \"kubernetes.io/projected/71057873-7e22-4597-a603-54d0ca0e2bfc-kube-api-access-mf5q2\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:30:15.391640 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.391639    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71057873-7e22-4597-a603-54d0ca0e2bfc-maas-api-tls\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:30:15.391782 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.391649    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71057873-7e22-4597-a603-54d0ca0e2bfc-openshift-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:30:15.438778 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.438722    2562 generic.go:356] "Generic (PLEG): container finished" podID="71057873-7e22-4597-a603-54d0ca0e2bfc" containerID="537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691" exitCode=0
Jul 24 18:30:15.438778 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.438767    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"
Jul 24 18:30:15.438924 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.438809    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr" event={"ID":"71057873-7e22-4597-a603-54d0ca0e2bfc","Type":"ContainerDied","Data":"537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691"}
Jul 24 18:30:15.438924 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.438862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr" event={"ID":"71057873-7e22-4597-a603-54d0ca0e2bfc","Type":"ContainerDied","Data":"1f138b6c4b4c129d03394b9d25426f08212d55c5cfe4e47c166e6bb4846ba7e9"}
Jul 24 18:30:15.438924 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.438902    2562 scope.go:117] "RemoveContainer" containerID="537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691"
Jul 24 18:30:15.449178 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.449158    2562 scope.go:117] "RemoveContainer" containerID="537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691"
Jul 24 18:30:15.449473 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:30:15.449445    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691\": container with ID starting with 537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691 not found: ID does not exist" containerID="537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691"
Jul 24 18:30:15.449545 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.449478    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691"} err="failed to get container status \"537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691\": rpc error: code = NotFound desc = could not find container \"537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691\": container with ID starting with 537d5c54f395321a0a77844b6d25ae215f763f7e32afaa00c786d4536aa35691 not found: ID does not exist"
Jul 24 18:30:15.470494 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.470466    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"]
Jul 24 18:30:15.473509 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:15.473489    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-378627-6bf7c96597-24lhr"]
Jul 24 18:30:16.193872 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:16.193844    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"]
Jul 24 18:30:16.198744 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:16.198720    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-378627-mhqrr"]
Jul 24 18:30:17.407981 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:17.407939    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71057873-7e22-4597-a603-54d0ca0e2bfc" path="/var/lib/kubelet/pods/71057873-7e22-4597-a603-54d0ca0e2bfc/volumes"
Jul 24 18:30:17.408488 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:30:17.408426    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b948496e-bdd0-4313-9e18-34f75ba5efca" path="/var/lib/kubelet/pods/b948496e-bdd0-4313-9e18-34f75ba5efca/volumes"
Jul 24 18:31:11.588675 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.588646    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"]
Jul 24 18:31:11.589114 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.589100    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14e3eec7-1d73-4d36-8720-b200d6c1e232" containerName="cleanup"
Jul 24 18:31:11.589114 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.589115    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="14e3eec7-1d73-4d36-8720-b200d6c1e232" containerName="cleanup"
Jul 24 18:31:11.589198 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.589125    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d1bb7e-88ac-482d-b3c7-9b7d3d267a84" containerName="cleanup"
Jul 24 18:31:11.589198 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.589130    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d1bb7e-88ac-482d-b3c7-9b7d3d267a84" containerName="cleanup"
Jul 24 18:31:11.589198 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.589143    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71057873-7e22-4597-a603-54d0ca0e2bfc" containerName="maas-api"
Jul 24 18:31:11.589198 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.589149    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="71057873-7e22-4597-a603-54d0ca0e2bfc" containerName="maas-api"
Jul 24 18:31:11.589324 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.589229    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="71057873-7e22-4597-a603-54d0ca0e2bfc" containerName="maas-api"
Jul 24 18:31:11.589324 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.589241    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="14e3eec7-1d73-4d36-8720-b200d6c1e232" containerName="cleanup"
Jul 24 18:31:11.589324 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.589249    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d1bb7e-88ac-482d-b3c7-9b7d3d267a84" containerName="cleanup"
Jul 24 18:31:11.592563 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.592545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"
Jul 24 18:31:11.595509 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.595487    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:31:11.599655 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.599619    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"]
Jul 24 18:31:11.760404 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.760374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e244e3-722f-4e23-abe2-921fba8bdaf9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g\" (UID: \"f6e244e3-722f-4e23-abe2-921fba8bdaf9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"
Jul 24 18:31:11.861449 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.861369    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e244e3-722f-4e23-abe2-921fba8bdaf9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g\" (UID: \"f6e244e3-722f-4e23-abe2-921fba8bdaf9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"
Jul 24 18:31:11.862006 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.861985    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e244e3-722f-4e23-abe2-921fba8bdaf9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g\" (UID: \"f6e244e3-722f-4e23-abe2-921fba8bdaf9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"
Jul 24 18:31:11.905391 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:11.905369    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"
Jul 24 18:31:12.049270 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:12.049243    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"]
Jul 24 18:31:12.049913 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:31:12.049868    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e244e3_722f_4e23_abe2_921fba8bdaf9.slice/crio-1363e99cb46c9863ac1b33a316b163dc9e791aa44e12b9e32d77d763136a9292 WatchSource:0}: Error finding container 1363e99cb46c9863ac1b33a316b163dc9e791aa44e12b9e32d77d763136a9292: Status 404 returned error can't find the container with id 1363e99cb46c9863ac1b33a316b163dc9e791aa44e12b9e32d77d763136a9292
Jul 24 18:31:12.801964 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:12.801935    2562 generic.go:356] "Generic (PLEG): container finished" podID="f6e244e3-722f-4e23-abe2-921fba8bdaf9" containerID="96383f963c16bd7ec422750b5b1d342737537fdf3919ba8b8ea18837c81accf6" exitCode=0
Jul 24 18:31:12.802244 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:12.802017    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g" event={"ID":"f6e244e3-722f-4e23-abe2-921fba8bdaf9","Type":"ContainerDied","Data":"96383f963c16bd7ec422750b5b1d342737537fdf3919ba8b8ea18837c81accf6"}
Jul 24 18:31:12.802244 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:12.802068    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g" event={"ID":"f6e244e3-722f-4e23-abe2-921fba8bdaf9","Type":"ContainerStarted","Data":"1363e99cb46c9863ac1b33a316b163dc9e791aa44e12b9e32d77d763136a9292"}
Jul 24 18:31:13.958147 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:13.958123    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"
Jul 24 18:31:14.081617 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:14.081558    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e244e3-722f-4e23-abe2-921fba8bdaf9-maas-api-service-ca\") pod \"f6e244e3-722f-4e23-abe2-921fba8bdaf9\" (UID: \"f6e244e3-722f-4e23-abe2-921fba8bdaf9\") "
Jul 24 18:31:14.081930 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:14.081911    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e244e3-722f-4e23-abe2-921fba8bdaf9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f6e244e3-722f-4e23-abe2-921fba8bdaf9" (UID: "f6e244e3-722f-4e23-abe2-921fba8bdaf9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:31:14.182689 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:14.182667    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e244e3-722f-4e23-abe2-921fba8bdaf9-maas-api-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:14.816019 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:14.815987    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"
Jul 24 18:31:14.816171 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:14.815992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g" event={"ID":"f6e244e3-722f-4e23-abe2-921fba8bdaf9","Type":"ContainerDied","Data":"1363e99cb46c9863ac1b33a316b163dc9e791aa44e12b9e32d77d763136a9292"}
Jul 24 18:31:14.816171 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:14.816112    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1363e99cb46c9863ac1b33a316b163dc9e791aa44e12b9e32d77d763136a9292"
Jul 24 18:31:24.170778 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.170744    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"]
Jul 24 18:31:24.171291 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.170998    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs" podUID="6b070106-deec-4658-b65b-260a6356fe12" containerName="maas-api" containerID="cri-o://319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504" gracePeriod=30
Jul 24 18:31:24.201114 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.201088    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6"]
Jul 24 18:31:24.206095 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.206072    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cb47da-29748630-x4tv6"]
Jul 24 18:31:24.426237 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.426180    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:31:24.463897 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.463851    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b070106-deec-4658-b65b-260a6356fe12-openshift-service-ca\") pod \"6b070106-deec-4658-b65b-260a6356fe12\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") "
Jul 24 18:31:24.464046 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.463983    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zszs\" (UniqueName: \"kubernetes.io/projected/6b070106-deec-4658-b65b-260a6356fe12-kube-api-access-2zszs\") pod \"6b070106-deec-4658-b65b-260a6356fe12\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") "
Jul 24 18:31:24.464046 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.464009    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b070106-deec-4658-b65b-260a6356fe12-maas-api-tls\") pod \"6b070106-deec-4658-b65b-260a6356fe12\" (UID: \"6b070106-deec-4658-b65b-260a6356fe12\") "
Jul 24 18:31:24.464265 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.464244    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b070106-deec-4658-b65b-260a6356fe12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b070106-deec-4658-b65b-260a6356fe12" (UID: "6b070106-deec-4658-b65b-260a6356fe12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:31:24.466733 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.466708    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b070106-deec-4658-b65b-260a6356fe12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b070106-deec-4658-b65b-260a6356fe12" (UID: "6b070106-deec-4658-b65b-260a6356fe12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:31:24.466733 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.466711    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b070106-deec-4658-b65b-260a6356fe12-kube-api-access-2zszs" (OuterVolumeSpecName: "kube-api-access-2zszs") pod "6b070106-deec-4658-b65b-260a6356fe12" (UID: "6b070106-deec-4658-b65b-260a6356fe12"). InnerVolumeSpecName "kube-api-access-2zszs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:31:24.564985 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.564955    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b070106-deec-4658-b65b-260a6356fe12-openshift-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:24.564985 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.564980    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zszs\" (UniqueName: \"kubernetes.io/projected/6b070106-deec-4658-b65b-260a6356fe12-kube-api-access-2zszs\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:24.564985 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.564990    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b070106-deec-4658-b65b-260a6356fe12-maas-api-tls\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:24.879031 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.879006    2562 generic.go:356] "Generic (PLEG): container finished" podID="6b070106-deec-4658-b65b-260a6356fe12" containerID="319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504" exitCode=0
Jul 24 18:31:24.879173 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.879048    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"
Jul 24 18:31:24.879173 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.879098    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs" event={"ID":"6b070106-deec-4658-b65b-260a6356fe12","Type":"ContainerDied","Data":"319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504"}
Jul 24 18:31:24.879173 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.879146    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs" event={"ID":"6b070106-deec-4658-b65b-260a6356fe12","Type":"ContainerDied","Data":"3bdee44c4d535f582eb6e0c456d2208bb5ef1c8b152869b85db2bdb461441c46"}
Jul 24 18:31:24.879173 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.879170    2562 scope.go:117] "RemoveContainer" containerID="319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504"
Jul 24 18:31:24.888772 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.888756    2562 scope.go:117] "RemoveContainer" containerID="319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504"
Jul 24 18:31:24.889068 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:31:24.889049    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504\": container with ID starting with 319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504 not found: ID does not exist" containerID="319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504"
Jul 24 18:31:24.889154 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.889074    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504"} err="failed to get container status \"319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504\": rpc error: code = NotFound desc = could not find container \"319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504\": container with ID starting with 319061ae31df92050dd1b25f606528c3baed31366136cbe131c83bd41c250504 not found: ID does not exist"
Jul 24 18:31:24.902278 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.902258    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"]
Jul 24 18:31:24.907961 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:24.907941    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb47da-5cdf566866-t7khs"]
Jul 24 18:31:25.409194 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:25.409165    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b070106-deec-4658-b65b-260a6356fe12" path="/var/lib/kubelet/pods/6b070106-deec-4658-b65b-260a6356fe12/volumes"
Jul 24 18:31:25.409639 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:25.409471    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77d1bb7e-88ac-482d-b3c7-9b7d3d267a84" path="/var/lib/kubelet/pods/77d1bb7e-88ac-482d-b3c7-9b7d3d267a84/volumes"
Jul 24 18:31:26.816904 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:26.816850    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"]
Jul 24 18:31:26.819336 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:26.819314    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb47da-69d8g"]
Jul 24 18:31:27.408786 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:27.408758    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e244e3-722f-4e23-abe2-921fba8bdaf9" path="/var/lib/kubelet/pods/f6e244e3-722f-4e23-abe2-921fba8bdaf9/volumes"
Jul 24 18:31:33.946300 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:33.946267    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"]
Jul 24 18:31:33.947336 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:33.947057    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy" containerID="cri-o://fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911" gracePeriod=30
Jul 24 18:31:34.420992 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.420959    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"]
Jul 24 18:31:34.421375 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.421363    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e244e3-722f-4e23-abe2-921fba8bdaf9" containerName="revoke-keys"
Jul 24 18:31:34.421433 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.421377    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e244e3-722f-4e23-abe2-921fba8bdaf9" containerName="revoke-keys"
Jul 24 18:31:34.421433 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.421390    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b070106-deec-4658-b65b-260a6356fe12" containerName="maas-api"
Jul 24 18:31:34.421433 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.421395    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b070106-deec-4658-b65b-260a6356fe12" containerName="maas-api"
Jul 24 18:31:34.421532 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.421485    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e244e3-722f-4e23-abe2-921fba8bdaf9" containerName="revoke-keys"
Jul 24 18:31:34.421532 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.421492    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b070106-deec-4658-b65b-260a6356fe12" containerName="maas-api"
Jul 24 18:31:34.424501 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.424480    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"
Jul 24 18:31:34.427074 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.427053    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:31:34.432483 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.432460    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"]
Jul 24 18:31:34.436347 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.436326    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77c7361e-a459-4417-80d5-1a4697a98a27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4\" (UID: \"77c7361e-a459-4417-80d5-1a4697a98a27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"
Jul 24 18:31:34.536901 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.536848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77c7361e-a459-4417-80d5-1a4697a98a27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4\" (UID: \"77c7361e-a459-4417-80d5-1a4697a98a27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"
Jul 24 18:31:34.537444 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.537425    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77c7361e-a459-4417-80d5-1a4697a98a27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4\" (UID: \"77c7361e-a459-4417-80d5-1a4697a98a27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"
Jul 24 18:31:34.738144 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.738056    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"
Jul 24 18:31:34.879397 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.879368    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"]
Jul 24 18:31:34.880678 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:31:34.880653    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c7361e_a459_4417_80d5_1a4697a98a27.slice/crio-fdaae6f735abdcf4466819298c6d6cb6a13a4503b6638b6bf5ac196e2f61977f WatchSource:0}: Error finding container fdaae6f735abdcf4466819298c6d6cb6a13a4503b6638b6bf5ac196e2f61977f: Status 404 returned error can't find the container with id fdaae6f735abdcf4466819298c6d6cb6a13a4503b6638b6bf5ac196e2f61977f
Jul 24 18:31:34.945379 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:34.945353    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4" event={"ID":"77c7361e-a459-4417-80d5-1a4697a98a27","Type":"ContainerStarted","Data":"fdaae6f735abdcf4466819298c6d6cb6a13a4503b6638b6bf5ac196e2f61977f"}
Jul 24 18:31:35.719145 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:35.719110    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 18:31:35.719505 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:35.719195    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 18:31:35.952676 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:35.952592    2562 generic.go:356] "Generic (PLEG): container finished" podID="77c7361e-a459-4417-80d5-1a4697a98a27" containerID="aeb6841c3b77fa04b0f1ae4e88bbc3c7138c6accfefb1fbfd79e4ac98295a4ea" exitCode=0
Jul 24 18:31:35.952825 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:35.952674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4" event={"ID":"77c7361e-a459-4417-80d5-1a4697a98a27","Type":"ContainerDied","Data":"aeb6841c3b77fa04b0f1ae4e88bbc3c7138c6accfefb1fbfd79e4ac98295a4ea"}
Jul 24 18:31:37.098469 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:37.098444    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"
Jul 24 18:31:37.160609 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:37.160579    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77c7361e-a459-4417-80d5-1a4697a98a27-maas-api-service-ca\") pod \"77c7361e-a459-4417-80d5-1a4697a98a27\" (UID: \"77c7361e-a459-4417-80d5-1a4697a98a27\") "
Jul 24 18:31:37.160978 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:37.160955    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77c7361e-a459-4417-80d5-1a4697a98a27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77c7361e-a459-4417-80d5-1a4697a98a27" (UID: "77c7361e-a459-4417-80d5-1a4697a98a27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:31:37.262151 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:37.262079    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77c7361e-a459-4417-80d5-1a4697a98a27-maas-api-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:37.966450 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:37.966417    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"
Jul 24 18:31:37.966634 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:37.966456    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4" event={"ID":"77c7361e-a459-4417-80d5-1a4697a98a27","Type":"ContainerDied","Data":"fdaae6f735abdcf4466819298c6d6cb6a13a4503b6638b6bf5ac196e2f61977f"}
Jul 24 18:31:37.966634 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:37.966510    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdaae6f735abdcf4466819298c6d6cb6a13a4503b6638b6bf5ac196e2f61977f"
Jul 24 18:31:39.212987 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.212938    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:31:39.277346 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277309    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-data\") pod \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") "
Jul 24 18:31:39.277346 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277353    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-envoy\") pod \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") "
Jul 24 18:31:39.277595 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277382    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-podinfo\") pod \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") "
Jul 24 18:31:39.277595 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277404    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-socket\") pod \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") "
Jul 24 18:31:39.277595 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277434    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwngv\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-kube-api-access-bwngv\") pod \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") "
Jul 24 18:31:39.277595 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277465    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-certs\") pod \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") "
Jul 24 18:31:39.277595 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277491    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istiod-ca-cert\") pod \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") "
Jul 24 18:31:39.277595 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277558    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-credential-socket\") pod \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") "
Jul 24 18:31:39.277952 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277606    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-token\") pod \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\" (UID: \"749b4c56-a3d4-40fd-a8cc-089901a6bfcb\") "
Jul 24 18:31:39.277952 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277804    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "749b4c56-a3d4-40fd-a8cc-089901a6bfcb" (UID: "749b4c56-a3d4-40fd-a8cc-089901a6bfcb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:31:39.277952 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277943    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-certs\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:39.278115 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277807    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "749b4c56-a3d4-40fd-a8cc-089901a6bfcb" (UID: "749b4c56-a3d4-40fd-a8cc-089901a6bfcb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:31:39.278115 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.278014    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "749b4c56-a3d4-40fd-a8cc-089901a6bfcb" (UID: "749b4c56-a3d4-40fd-a8cc-089901a6bfcb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:31:39.278115 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.277956    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "749b4c56-a3d4-40fd-a8cc-089901a6bfcb" (UID: "749b4c56-a3d4-40fd-a8cc-089901a6bfcb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:31:39.278288 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.278262    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-data" (OuterVolumeSpecName: "istio-data") pod "749b4c56-a3d4-40fd-a8cc-089901a6bfcb" (UID: "749b4c56-a3d4-40fd-a8cc-089901a6bfcb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:31:39.279701 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.279668    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "749b4c56-a3d4-40fd-a8cc-089901a6bfcb" (UID: "749b4c56-a3d4-40fd-a8cc-089901a6bfcb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:31:39.279797 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.279718    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-kube-api-access-bwngv" (OuterVolumeSpecName: "kube-api-access-bwngv") pod "749b4c56-a3d4-40fd-a8cc-089901a6bfcb" (UID: "749b4c56-a3d4-40fd-a8cc-089901a6bfcb"). InnerVolumeSpecName "kube-api-access-bwngv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:31:39.279797 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.279728    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-token" (OuterVolumeSpecName: "istio-token") pod "749b4c56-a3d4-40fd-a8cc-089901a6bfcb" (UID: "749b4c56-a3d4-40fd-a8cc-089901a6bfcb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:31:39.279989 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.279973    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "749b4c56-a3d4-40fd-a8cc-089901a6bfcb" (UID: "749b4c56-a3d4-40fd-a8cc-089901a6bfcb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 18:31:39.378864 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.378834    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-credential-socket\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:39.378864 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.378859    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-token\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:39.378864 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.378868    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-data\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:39.379094 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.378899    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-envoy\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:39.379094 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.378911    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istio-podinfo\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:39.379094 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.378923    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-workload-socket\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:39.379094 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.378935    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwngv\" (UniqueName: \"kubernetes.io/projected/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-kube-api-access-bwngv\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:39.379094 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.378949    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749b4c56-a3d4-40fd-a8cc-089901a6bfcb-istiod-ca-cert\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:31:39.980794 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.980761    2562 generic.go:356] "Generic (PLEG): container finished" podID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerID="fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911" exitCode=0
Jul 24 18:31:39.980993 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.980821    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"
Jul 24 18:31:39.980993 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.980849    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" event={"ID":"749b4c56-a3d4-40fd-a8cc-089901a6bfcb","Type":"ContainerDied","Data":"fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911"}
Jul 24 18:31:39.980993 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.980918    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m" event={"ID":"749b4c56-a3d4-40fd-a8cc-089901a6bfcb","Type":"ContainerDied","Data":"50c61792f6a551133303190aabd2b0cef6ec2edcc10940e5b4d2a5a5077a31e5"}
Jul 24 18:31:39.980993 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.980942    2562 scope.go:117] "RemoveContainer" containerID="fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911"
Jul 24 18:31:39.990309 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.990293    2562 scope.go:117] "RemoveContainer" containerID="694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0"
Jul 24 18:31:39.998548 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.998532    2562 scope.go:117] "RemoveContainer" containerID="fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911"
Jul 24 18:31:39.998801 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:31:39.998785    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911\": container with ID starting with fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911 not found: ID does not exist" containerID="fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911"
Jul 24 18:31:39.998856 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.998808    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911"} err="failed to get container status \"fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911\": rpc error: code = NotFound desc = could not find container \"fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911\": container with ID starting with fef0609f8564bc6ab8bbe7e59a38b692b087af7e408336c31864f5e59a604911 not found: ID does not exist"
Jul 24 18:31:39.998856 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.998823    2562 scope.go:117] "RemoveContainer" containerID="694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0"
Jul 24 18:31:39.999052 ip-10-0-136-89 kubenswrapper[2562]: E0724 18:31:39.999033    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0\": container with ID starting with 694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0 not found: ID does not exist" containerID="694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0"
Jul 24 18:31:39.999109 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:39.999061    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0"} err="failed to get container status \"694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0\": rpc error: code = NotFound desc = could not find container \"694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0\": container with ID starting with 694c7c866e0e72723b37fb7f8f90a215e055df9f71a8ef30fc90ea989c3615f0 not found: ID does not exist"
Jul 24 18:31:40.005124 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:40.005102    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"]
Jul 24 18:31:40.008766 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:40.008746    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb47da-openshift-default-6db7895578-j8s8m"]
Jul 24 18:31:41.408694 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:41.408654    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" path="/var/lib/kubelet/pods/749b4c56-a3d4-40fd-a8cc-089901a6bfcb/volumes"
Jul 24 18:31:49.606578 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:49.606544    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"]
Jul 24 18:31:49.609035 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:49.609013    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b90928-2wlk4"]
Jul 24 18:31:51.407653 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:51.407614    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77c7361e-a459-4417-80d5-1a4697a98a27" path="/var/lib/kubelet/pods/77c7361e-a459-4417-80d5-1a4697a98a27/volumes"
Jul 24 18:31:57.152928 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.152886    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"]
Jul 24 18:31:57.153410 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.153299    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy"
Jul 24 18:31:57.153410 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.153309    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy"
Jul 24 18:31:57.153410 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.153333    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c7361e-a459-4417-80d5-1a4697a98a27" containerName="revoke-keys"
Jul 24 18:31:57.153410 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.153338    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c7361e-a459-4417-80d5-1a4697a98a27" containerName="revoke-keys"
Jul 24 18:31:57.153410 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.153399    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy"
Jul 24 18:31:57.153410 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.153407    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77c7361e-a459-4417-80d5-1a4697a98a27" containerName="revoke-keys"
Jul 24 18:31:57.153410 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.153422    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy"
Jul 24 18:31:57.158024 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.158008    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"
Jul 24 18:31:57.160893 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.160853    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\""
Jul 24 18:31:57.163179 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.163157    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"]
Jul 24 18:31:57.228861 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.228838    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2ab74f-1b04-4010-9616-a2bb90c8cfa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22\" (UID: \"7f2ab74f-1b04-4010-9616-a2bb90c8cfa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"
Jul 24 18:31:57.329281 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.329257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2ab74f-1b04-4010-9616-a2bb90c8cfa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22\" (UID: \"7f2ab74f-1b04-4010-9616-a2bb90c8cfa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"
Jul 24 18:31:57.329820 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.329803    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2ab74f-1b04-4010-9616-a2bb90c8cfa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22\" (UID: \"7f2ab74f-1b04-4010-9616-a2bb90c8cfa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"
Jul 24 18:31:57.470056 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.470007    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"
Jul 24 18:31:57.807433 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:57.807405    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"]
Jul 24 18:31:57.809211 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:31:57.809186    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2ab74f_1b04_4010_9616_a2bb90c8cfa0.slice/crio-4231af6f944111cbf089b5aca3857066aa8708840d209a2c08ba55d8a015d164 WatchSource:0}: Error finding container 4231af6f944111cbf089b5aca3857066aa8708840d209a2c08ba55d8a015d164: Status 404 returned error can't find the container with id 4231af6f944111cbf089b5aca3857066aa8708840d209a2c08ba55d8a015d164
Jul 24 18:31:58.096390 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:58.096359    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22" event={"ID":"7f2ab74f-1b04-4010-9616-a2bb90c8cfa0","Type":"ContainerStarted","Data":"4231af6f944111cbf089b5aca3857066aa8708840d209a2c08ba55d8a015d164"}
Jul 24 18:31:59.104371 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:59.104327    2562 generic.go:356] "Generic (PLEG): container finished" podID="7f2ab74f-1b04-4010-9616-a2bb90c8cfa0" containerID="8e49af41a7388b6b966bffac32d906677811da4a0111cfe50451d870a27f27ff" exitCode=0
Jul 24 18:31:59.104777 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:31:59.104411    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22" event={"ID":"7f2ab74f-1b04-4010-9616-a2bb90c8cfa0","Type":"ContainerDied","Data":"8e49af41a7388b6b966bffac32d906677811da4a0111cfe50451d870a27f27ff"}
Jul 24 18:32:00.251766 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:00.251739    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"
Jul 24 18:32:00.356302 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:00.356276    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2ab74f-1b04-4010-9616-a2bb90c8cfa0-maas-api-service-ca\") pod \"7f2ab74f-1b04-4010-9616-a2bb90c8cfa0\" (UID: \"7f2ab74f-1b04-4010-9616-a2bb90c8cfa0\") "
Jul 24 18:32:00.356646 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:00.356622    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2ab74f-1b04-4010-9616-a2bb90c8cfa0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f2ab74f-1b04-4010-9616-a2bb90c8cfa0" (UID: "7f2ab74f-1b04-4010-9616-a2bb90c8cfa0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:32:00.457196 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:00.457136    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2ab74f-1b04-4010-9616-a2bb90c8cfa0-maas-api-service-ca\") on node \"ip-10-0-136-89.ec2.internal\" DevicePath \"\""
Jul 24 18:32:01.118495 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:01.118463    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22" event={"ID":"7f2ab74f-1b04-4010-9616-a2bb90c8cfa0","Type":"ContainerDied","Data":"4231af6f944111cbf089b5aca3857066aa8708840d209a2c08ba55d8a015d164"}
Jul 24 18:32:01.118495 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:01.118484    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"
Jul 24 18:32:01.118747 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:01.118509    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4231af6f944111cbf089b5aca3857066aa8708840d209a2c08ba55d8a015d164"
Jul 24 18:32:12.264977 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:12.264939    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"]
Jul 24 18:32:12.270489 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:12.270464    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-88d4a7-9qv22"]
Jul 24 18:32:13.408944 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:13.408906    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f2ab74f-1b04-4010-9616-a2bb90c8cfa0" path="/var/lib/kubelet/pods/7f2ab74f-1b04-4010-9616-a2bb90c8cfa0/volumes"
Jul 24 18:32:19.902804 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:19.902774    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65bfbcb984-gjmwg_80f83c3e-d8e1-465a-8d9e-a471a04c72e9/authorino/0.log"
Jul 24 18:32:25.694072 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:25.694038    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2jnjw_c087d267-61f1-4796-88d5-1564dfe417fc/manager/0.log"
Jul 24 18:32:25.819613 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:25.819583    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fx66p_fb9c57db-3607-4779-8af3-91fde3a74002/manager/0.log"
Jul 24 18:32:26.124133 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:26.124109    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cdhx8_935e18f3-2fcc-4d93-91fa-da3b7f510aa9/server/0.log"
Jul 24 18:32:26.251611 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:26.251571    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gqrlz_d6ec55d0-2bd3-4922-8f5e-0576651571e1/manager/0.log"
Jul 24 18:32:26.497299 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:26.497236    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f579866-ftf6f_9d5925b1-46b5-4431-a612-bf3010884213/copy-manifests/0.log"
Jul 24 18:32:26.505183 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:26.505157    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f579866-ftf6f_9d5925b1-46b5-4431-a612-bf3010884213/manager/0.log"
Jul 24 18:32:27.023709 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:27.023681    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5c8f99fbb4-kr6kk_72e72bfd-0350-4439-ac35-6cd5bd27f6ff/maas-api/0.log"
Jul 24 18:32:27.282395 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:27.282317    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748615-f6z62_538b26ac-2494-4c0d-a62d-6e04c3a56bce/cleanup/0.log"
Jul 24 18:32:27.401565 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:27.401544    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748630-5d8nb_14e3eec7-1d73-4d36-8720-b200d6c1e232/cleanup/0.log"
Jul 24 18:32:27.528265 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:27.528242    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-sjlz2_ae4ad542-6006-4ce2-9ae1-8f0b092ccc08/postgres/0.log"
Jul 24 18:32:28.367286 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.367257    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr_311055f3-f587-4d42-a8bc-e549b2817d89/util/0.log"
Jul 24 18:32:28.373652 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.373631    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr_311055f3-f587-4d42-a8bc-e549b2817d89/pull/0.log"
Jul 24 18:32:28.380025 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.379998    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr_311055f3-f587-4d42-a8bc-e549b2817d89/extract/0.log"
Jul 24 18:32:28.512566 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.512540    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9_897af391-380d-4470-9f9b-297c14483c2a/util/0.log"
Jul 24 18:32:28.519988 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.519968    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9_897af391-380d-4470-9f9b-297c14483c2a/pull/0.log"
Jul 24 18:32:28.527428 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.527409    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9_897af391-380d-4470-9f9b-297c14483c2a/extract/0.log"
Jul 24 18:32:28.653599 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.653549    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw_66aa18ba-854e-498e-8469-4721f4664c8e/util/0.log"
Jul 24 18:32:28.660679 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.660663    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw_66aa18ba-854e-498e-8469-4721f4664c8e/pull/0.log"
Jul 24 18:32:28.669140 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.669123    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw_66aa18ba-854e-498e-8469-4721f4664c8e/extract/0.log"
Jul 24 18:32:28.791120 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.791100    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84_9bc9c580-1d34-4ee0-b364-173f8bc862b7/util/0.log"
Jul 24 18:32:28.797200 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.797182    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84_9bc9c580-1d34-4ee0-b364-173f8bc862b7/pull/0.log"
Jul 24 18:32:28.803244 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.803227    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84_9bc9c580-1d34-4ee0-b364-173f8bc862b7/extract/0.log"
Jul 24 18:32:28.933049 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:28.932998    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65bfbcb984-gjmwg_80f83c3e-d8e1-465a-8d9e-a471a04c72e9/authorino/0.log"
Jul 24 18:32:29.067176 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:29.067154    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-b5n7m_d8ee1ef8-7261-434a-a6ee-0a7afe24011e/manager/0.log"
Jul 24 18:32:29.184705 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:29.184654    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lnpmv_f43329ad-5912-4f77-9177-e70f7089b65e/manager/0.log"
Jul 24 18:32:29.312088 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:29.312070    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rsxj7_452d88e5-953a-4211-8a5f-2eef3c1c7a8a/kuadrant-console-plugin/0.log"
Jul 24 18:32:29.434661 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:29.434634    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-56vqg_91411414-1e88-46dc-bf85-1c1f5d6d146a/registry-server/0.log"
Jul 24 18:32:29.944574 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:29.944548    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-x7rw4_11407094-b4c8-4d0b-a597-e11f04ef6ec1/manager/0.log"
Jul 24 18:32:30.606061 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:30.606038    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69ddb4868f-2bngz_d58dc331-b5f6-4522-82fd-c56eaa0f150e/kube-auth-proxy/0.log"
Jul 24 18:32:30.862855 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:30.862779    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qbgk5_391b8927-4198-458a-92b4-e3528d751abe/istio-proxy/1.log"
Jul 24 18:32:31.005020 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:31.004996    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-wll8r_2299da18-3500-4fa7-bb76-dd8ad6086623/payload-pre-processing/0.log"
Jul 24 18:32:31.135392 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:31.135330    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w5rtf_1e1e7f00-128d-419c-8297-673e02231338/payload-processing/0.log"
Jul 24 18:32:31.264801 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:31.264760    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5cb6869597-2kxmd_2f5568f4-e4a5-4007-adb7-780b43c081ad/router/0.log"
Jul 24 18:32:31.638566 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:31.638538    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lzbs6_ae657de7-9977-4f12-8346-5e5df2c79355/main/0.log"
Jul 24 18:32:32.001603 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:32.001543    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb7rgd_f1ccc6e4-341d-4e7c-ae8e-2bf6afa957cd/main/0.log"
Jul 24 18:32:40.824341 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:40.824303    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8fsnd_74b9bf45-8bba-459e-bd93-ee92f990ae95/global-pull-secret-syncer/0.log"
Jul 24 18:32:40.988415 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:40.988386    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p9kck_aec7a6de-7be2-41cc-b3f1-d447f0bbdab3/konnectivity-agent/0.log"
Jul 24 18:32:41.045357 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:41.045333    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-89.ec2.internal_dd13083ddd3ee48db23d4b5c37fef9a4/haproxy/0.log"
Jul 24 18:32:45.469600 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.469517    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr_311055f3-f587-4d42-a8bc-e549b2817d89/extract/0.log"
Jul 24 18:32:45.492372 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.492354    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr_311055f3-f587-4d42-a8bc-e549b2817d89/util/0.log"
Jul 24 18:32:45.521444 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.521422    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmrrr_311055f3-f587-4d42-a8bc-e549b2817d89/pull/0.log"
Jul 24 18:32:45.555924 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.555902    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9_897af391-380d-4470-9f9b-297c14483c2a/extract/0.log"
Jul 24 18:32:45.580945 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.580930    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9_897af391-380d-4470-9f9b-297c14483c2a/util/0.log"
Jul 24 18:32:45.601717 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.601700    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vfzh9_897af391-380d-4470-9f9b-297c14483c2a/pull/0.log"
Jul 24 18:32:45.628462 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.628446    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw_66aa18ba-854e-498e-8469-4721f4664c8e/extract/0.log"
Jul 24 18:32:45.652506 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.652486    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw_66aa18ba-854e-498e-8469-4721f4664c8e/util/0.log"
Jul 24 18:32:45.675393 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.675370    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mxwcw_66aa18ba-854e-498e-8469-4721f4664c8e/pull/0.log"
Jul 24 18:32:45.704706 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.704692    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84_9bc9c580-1d34-4ee0-b364-173f8bc862b7/extract/0.log"
Jul 24 18:32:45.729421 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.729366    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84_9bc9c580-1d34-4ee0-b364-173f8bc862b7/util/0.log"
Jul 24 18:32:45.753993 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:45.753976    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f9c84_9bc9c580-1d34-4ee0-b364-173f8bc862b7/pull/0.log"
Jul 24 18:32:46.081495 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:46.081470    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65bfbcb984-gjmwg_80f83c3e-d8e1-465a-8d9e-a471a04c72e9/authorino/0.log"
Jul 24 18:32:46.119449 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:46.119421    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-b5n7m_d8ee1ef8-7261-434a-a6ee-0a7afe24011e/manager/0.log"
Jul 24 18:32:46.157330 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:46.157301    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lnpmv_f43329ad-5912-4f77-9177-e70f7089b65e/manager/0.log"
Jul 24 18:32:46.202578 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:46.202556    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rsxj7_452d88e5-953a-4211-8a5f-2eef3c1c7a8a/kuadrant-console-plugin/0.log"
Jul 24 18:32:46.253323 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:46.253303    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-56vqg_91411414-1e88-46dc-bf85-1c1f5d6d146a/registry-server/0.log"
Jul 24 18:32:46.504283 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:46.504216    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-x7rw4_11407094-b4c8-4d0b-a597-e11f04ef6ec1/manager/0.log"
Jul 24 18:32:48.580810 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:48.580781    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-kzcs4_a9411acd-1297-42d0-8b2e-bc84a5cfec3b/cluster-monitoring-operator/0.log"
Jul 24 18:32:48.859034 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:48.858967    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8jqms_35eb8864-a07c-4198-951e-d240bd5036be/node-exporter/0.log"
Jul 24 18:32:48.880405 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:48.880385    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8jqms_35eb8864-a07c-4198-951e-d240bd5036be/kube-rbac-proxy/0.log"
Jul 24 18:32:48.902737 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:48.902717    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8jqms_35eb8864-a07c-4198-951e-d240bd5036be/init-textfile/0.log"
Jul 24 18:32:49.379460 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:49.379435    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-qgzlf_c421f0aa-be5d-4fa9-8b3a-40db518783a3/prometheus-operator-admission-webhook/0.log"
Jul 24 18:32:50.805188 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:50.805148    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-s8skq_03625b16-050a-4640-8f1b-b8071d53cdc8/networking-console-plugin/0.log"
Jul 24 18:32:51.200592 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.200506    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"]
Jul 24 18:32:51.200968 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.200953    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy"
Jul 24 18:32:51.200968 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.200969    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="749b4c56-a3d4-40fd-a8cc-089901a6bfcb" containerName="istio-proxy"
Jul 24 18:32:51.201078 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.200993    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2ab74f-1b04-4010-9616-a2bb90c8cfa0" containerName="revoke-keys"
Jul 24 18:32:51.201078 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.200999    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2ab74f-1b04-4010-9616-a2bb90c8cfa0" containerName="revoke-keys"
Jul 24 18:32:51.201078 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.201066    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f2ab74f-1b04-4010-9616-a2bb90c8cfa0" containerName="revoke-keys"
Jul 24 18:32:51.204712 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.204690    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.207267 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.207169    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dhhl7\"/\"kube-root-ca.crt\""
Jul 24 18:32:51.207267 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.207219    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dhhl7\"/\"openshift-service-ca.crt\""
Jul 24 18:32:51.208229 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.208207    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dhhl7\"/\"default-dockercfg-hrkj5\""
Jul 24 18:32:51.211693 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.211633    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"]
Jul 24 18:32:51.256318 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.256295    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-podres\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.256420 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.256334    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-sys\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.256477 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.256416    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-proc\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.256525 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.256476    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-lib-modules\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.256525 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.256501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwfvq\" (UniqueName: \"kubernetes.io/projected/a9859dac-b225-41e5-b589-fac76a8c006a-kube-api-access-hwfvq\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.357506 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.357483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-podres\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.357612 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.357514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-sys\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.357612 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.357544    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-proc\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.357612 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.357575    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-lib-modules\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.357612 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.357594    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwfvq\" (UniqueName: \"kubernetes.io/projected/a9859dac-b225-41e5-b589-fac76a8c006a-kube-api-access-hwfvq\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.357789 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.357625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-proc\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.357789 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.357625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-sys\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.357789 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.357664    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-podres\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.357789 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.357713    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9859dac-b225-41e5-b589-fac76a8c006a-lib-modules\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.365769 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.365745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwfvq\" (UniqueName: \"kubernetes.io/projected/a9859dac-b225-41e5-b589-fac76a8c006a-kube-api-access-hwfvq\") pod \"perf-node-gather-daemonset-9sdfw\" (UID: \"a9859dac-b225-41e5-b589-fac76a8c006a\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.471063 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.471005    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/2.log"
Jul 24 18:32:51.475091 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.475074    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8gfzp_556df0ae-3850-45c5-a889-d36921adf671/console-operator/3.log"
Jul 24 18:32:51.516553 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.516530    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:51.655392 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:51.655369    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"]
Jul 24 18:32:51.656863 ip-10-0-136-89 kubenswrapper[2562]: W0724 18:32:51.656838    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda9859dac_b225_41e5_b589_fac76a8c006a.slice/crio-7fd11ed36428f26fdf872d1ee488174d78bd9147d4976fd559ce2fea9215f7b9 WatchSource:0}: Error finding container 7fd11ed36428f26fdf872d1ee488174d78bd9147d4976fd559ce2fea9215f7b9: Status 404 returned error can't find the container with id 7fd11ed36428f26fdf872d1ee488174d78bd9147d4976fd559ce2fea9215f7b9
Jul 24 18:32:52.012747 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:52.012669    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f5bf9689b-ztkgv_a1f02c7e-e4b1-4cb1-bc4c-6117993c9d38/console/0.log"
Jul 24 18:32:52.048090 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:52.048067    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-6s4h4_2ccc92be-bd5c-41c7-84e9-6b2e76f71654/download-server/0.log"
Jul 24 18:32:52.436852 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:52.436821    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw" event={"ID":"a9859dac-b225-41e5-b589-fac76a8c006a","Type":"ContainerStarted","Data":"0be50080f9b02e8733cf894bc13de9350ff1baa90b3a92419fd254f6d357cddd"}
Jul 24 18:32:52.437025 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:52.436871    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw" event={"ID":"a9859dac-b225-41e5-b589-fac76a8c006a","Type":"ContainerStarted","Data":"7fd11ed36428f26fdf872d1ee488174d78bd9147d4976fd559ce2fea9215f7b9"}
Jul 24 18:32:52.437025 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:52.436978    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:52.458711 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:52.458665    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw" podStartSLOduration=1.458652032 podStartE2EDuration="1.458652032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:32:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:32:52.45804114 +0000 UTC m=+3179.662733021" watchObservedRunningTime="2026-07-24 18:32:52.458652032 +0000 UTC m=+3179.663343829"
Jul 24 18:32:52.644312 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:52.644288    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-db627_b3cb69d8-c6ec-4537-aae4-d202504f2fc8/volume-data-source-validator/0.log"
Jul 24 18:32:53.743599 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:53.743567    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7jczg_d93ed42c-d048-484d-941a-93193f1e13cd/dns/0.log"
Jul 24 18:32:53.764278 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:53.764261    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7jczg_d93ed42c-d048-484d-941a-93193f1e13cd/kube-rbac-proxy/0.log"
Jul 24 18:32:53.887262 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:53.887237    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t4rbd_291c612c-f9bd-4525-a4b6-e3f6777def41/dns-node-resolver/0.log"
Jul 24 18:32:54.485754 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:54.485727    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t4t99_81d447a3-e3df-4850-bcd4-60ef6e80cbcd/node-ca/0.log"
Jul 24 18:32:55.594622 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:55.594591    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69ddb4868f-2bngz_d58dc331-b5f6-4522-82fd-c56eaa0f150e/kube-auth-proxy/0.log"
Jul 24 18:32:55.656349 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:55.656315    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qbgk5_391b8927-4198-458a-92b4-e3528d751abe/istio-proxy/0.log"
Jul 24 18:32:55.682981 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:55.682939    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qbgk5_391b8927-4198-458a-92b4-e3528d751abe/istio-proxy/1.log"
Jul 24 18:32:55.714635 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:55.714614    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-wll8r_2299da18-3500-4fa7-bb76-dd8ad6086623/payload-pre-processing/0.log"
Jul 24 18:32:55.747594 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:55.747573    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w5rtf_1e1e7f00-128d-419c-8297-673e02231338/payload-processing/0.log"
Jul 24 18:32:55.770555 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:55.770529    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5cb6869597-2kxmd_2f5568f4-e4a5-4007-adb7-780b43c081ad/router/0.log"
Jul 24 18:32:56.638774 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:56.638747    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v622r_cc2d1617-bb81-4541-9971-09854abe6150/serve-healthcheck-canary/0.log"
Jul 24 18:32:57.122483 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:57.122456    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-m7799_d66bc6dd-82ce-473f-8aba-de7d3929b6bb/insights-operator/0.log"
Jul 24 18:32:57.123353 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:57.123334    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-m7799_d66bc6dd-82ce-473f-8aba-de7d3929b6bb/insights-operator/1.log"
Jul 24 18:32:57.226978 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:57.226955    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xcppd_2ae081de-f739-45a4-a66e-7f9d9e56f35b/kube-rbac-proxy/0.log"
Jul 24 18:32:57.248523 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:57.248505    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xcppd_2ae081de-f739-45a4-a66e-7f9d9e56f35b/exporter/0.log"
Jul 24 18:32:57.271212 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:57.271191    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xcppd_2ae081de-f739-45a4-a66e-7f9d9e56f35b/extractor/0.log"
Jul 24 18:32:58.453272 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:58.453246    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-9sdfw"
Jul 24 18:32:59.413841 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:59.413808    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2jnjw_c087d267-61f1-4796-88d5-1564dfe417fc/manager/0.log"
Jul 24 18:32:59.606356 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:59.606329    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fx66p_fb9c57db-3607-4779-8af3-91fde3a74002/manager/0.log"
Jul 24 18:32:59.939323 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:32:59.939295    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cdhx8_935e18f3-2fcc-4d93-91fa-da3b7f510aa9/server/0.log"
Jul 24 18:33:00.019962 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:00.019936    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gqrlz_d6ec55d0-2bd3-4922-8f5e-0576651571e1/manager/0.log"
Jul 24 18:33:00.101192 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:00.101173    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f579866-ftf6f_9d5925b1-46b5-4431-a612-bf3010884213/manager/0.log"
Jul 24 18:33:00.125722 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:00.125704    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f579866-ftf6f_9d5925b1-46b5-4431-a612-bf3010884213/copy-manifests/0.log"
Jul 24 18:33:01.729941 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:01.729860    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ld5t2_4ed36f70-2113-40ee-a001-5f52a07f7cdc/openshift-lws-operator/0.log"
Jul 24 18:33:01.738456 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:01.738437    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69ddb4868f-2bngz_d58dc331-b5f6-4522-82fd-c56eaa0f150e/kube-auth-proxy/0.log"
Jul 24 18:33:01.772936 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:01.772916    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qbgk5_391b8927-4198-458a-92b4-e3528d751abe/istio-proxy/0.log"
Jul 24 18:33:01.796553 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:01.796530    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qbgk5_391b8927-4198-458a-92b4-e3528d751abe/istio-proxy/1.log"
Jul 24 18:33:01.816558 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:01.816542    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-wll8r_2299da18-3500-4fa7-bb76-dd8ad6086623/payload-pre-processing/0.log"
Jul 24 18:33:01.836419 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:01.836398    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w5rtf_1e1e7f00-128d-419c-8297-673e02231338/payload-processing/0.log"
Jul 24 18:33:01.850990 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:01.850968    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5cb6869597-2kxmd_2f5568f4-e4a5-4007-adb7-780b43c081ad/router/0.log"
Jul 24 18:33:03.397407 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:03.397383    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69ddb4868f-2bngz_d58dc331-b5f6-4522-82fd-c56eaa0f150e/kube-auth-proxy/0.log"
Jul 24 18:33:03.438175 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:03.438150    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qbgk5_391b8927-4198-458a-92b4-e3528d751abe/istio-proxy/0.log"
Jul 24 18:33:03.466041 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:03.466020    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qbgk5_391b8927-4198-458a-92b4-e3528d751abe/istio-proxy/1.log"
Jul 24 18:33:03.486688 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:03.486667    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-wll8r_2299da18-3500-4fa7-bb76-dd8ad6086623/payload-pre-processing/0.log"
Jul 24 18:33:03.506662 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:03.506640    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w5rtf_1e1e7f00-128d-419c-8297-673e02231338/payload-processing/0.log"
Jul 24 18:33:03.519854 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:03.519834    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5cb6869597-2kxmd_2f5568f4-e4a5-4007-adb7-780b43c081ad/router/0.log"
Jul 24 18:33:07.024552 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:07.024521    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q_8dd06a5f-d5af-4e05-bd87-afdbf92c7b43/kube-storage-version-migrator-operator/1.log"
Jul 24 18:33:07.025299 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:07.025280    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-trb8q_8dd06a5f-d5af-4e05-bd87-afdbf92c7b43/kube-storage-version-migrator-operator/0.log"
Jul 24 18:33:08.415198 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.415169    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tzghr_c4707a45-429b-4d07-a7b9-41d3bcf570fa/kube-multus-additional-cni-plugins/0.log"
Jul 24 18:33:08.436917 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.436896    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tzghr_c4707a45-429b-4d07-a7b9-41d3bcf570fa/egress-router-binary-copy/0.log"
Jul 24 18:33:08.458583 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.458565    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tzghr_c4707a45-429b-4d07-a7b9-41d3bcf570fa/cni-plugins/0.log"
Jul 24 18:33:08.483192 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.483176    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tzghr_c4707a45-429b-4d07-a7b9-41d3bcf570fa/bond-cni-plugin/0.log"
Jul 24 18:33:08.511449 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.511434    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tzghr_c4707a45-429b-4d07-a7b9-41d3bcf570fa/routeoverride-cni/0.log"
Jul 24 18:33:08.533848 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.533831    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tzghr_c4707a45-429b-4d07-a7b9-41d3bcf570fa/whereabouts-cni-bincopy/0.log"
Jul 24 18:33:08.556551 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.556535    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tzghr_c4707a45-429b-4d07-a7b9-41d3bcf570fa/whereabouts-cni/0.log"
Jul 24 18:33:08.656198 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.656177    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pl7lv_5a8021bc-a6d1-4d9b-be64-b04494e23e90/kube-multus/0.log"
Jul 24 18:33:08.727927 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.727848    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kz48t_64665c61-db20-4439-a490-af90a1d0e579/network-metrics-daemon/0.log"
Jul 24 18:33:08.746018 ip-10-0-136-89 kubenswrapper[2562]: I0724 18:33:08.745980    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kz48t_64665c61-db20-4439-a490-af90a1d0e579/kube-rbac-proxy/0.log"