Aug 02 10:33:18.362402 ip-10-0-135-193 systemd[1]: Starting Kubernetes Kubelet...
Aug 02 10:33:18.722913 ip-10-0-135-193 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 02 10:33:18.722913 ip-10-0-135-193 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 02 10:33:18.722913 ip-10-0-135-193 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 02 10:33:18.722913 ip-10-0-135-193 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 02 10:33:18.722913 ip-10-0-135-193 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 02 10:33:18.724916 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.724806    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 02 10:33:18.728153 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728139    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 02 10:33:18.728153 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728153    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728156    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728160    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728163    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728166    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728169    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728172    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728175    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728178    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728181    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728189    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728191    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728194    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728196    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728199    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728201    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728204    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728206    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728209    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728211    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 02 10:33:18.728222 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728214    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728216    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728219    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728221    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728224    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728227    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728229    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728232    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728237    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728240    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728243    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728246    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728249    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728252    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728255    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728258    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728261    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728263    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728266    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728268    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 02 10:33:18.728705 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728270    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728273    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728275    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728278    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728281    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728283    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728285    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728288    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728290    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728293    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728295    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728298    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728301    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728303    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728307    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728310    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728312    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728315    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728317    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728319    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 02 10:33:18.729204 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728322    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728324    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728327    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728329    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728332    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728334    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728337    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728339    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728342    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728344    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728347    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728349    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728352    2576 feature_gate.go:328] unrecognized feature gate: Example2
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728354    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728357    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728360    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728362    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728365    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728367    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728370    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 02 10:33:18.729692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728372    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728375    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728377    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728379    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728382    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728387    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728391    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728394    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728397    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728400    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728402    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728405    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728408    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728411    2576 feature_gate.go:328] unrecognized feature gate: Example
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728415    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728418    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728422    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.728424    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729189    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 02 10:33:18.730195 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729197    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729200    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729203    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729206    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729209    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729212    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729215    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729218    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729220    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729223    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729226    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729229    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729231    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729234    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729237    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729239    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729241    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729244    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729246    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729251    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 02 10:33:18.730647 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729253    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729256    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729259    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729262    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729265    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729267    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729270    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729272    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729275    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729277    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729279    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729282    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729284    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729287    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729289    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729292    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729294    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729296    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729299    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729301    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 02 10:33:18.731355 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729304    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729306    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729309    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729312    2576 feature_gate.go:328] unrecognized feature gate: Example
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729314    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729318    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729322    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729325    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729327    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729330    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729332    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729335    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729338    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729340    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729343    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729345    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729348    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729350    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729352    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729355    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 02 10:33:18.732152 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729357    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729359    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729362    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729364    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729367    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729369    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729371    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729374    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729376    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729379    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729381    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729383    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729386    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729388    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729391    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729393    2576 feature_gate.go:328] unrecognized feature gate: Example2
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729396    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729400    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729404    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 02 10:33:18.732661 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729406    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729411    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729414    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729417    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729420    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729423    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729425    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729429    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729431    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729434    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729436    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729439    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729441    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729443    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729446    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729448    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729451    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729453    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729455    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 02 10:33:18.733221 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729824    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729828    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729831    2576 feature_gate.go:328] unrecognized feature gate: Example
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729833    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729836    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729839    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729841    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729844    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729846    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729848    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729851    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729854    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729856    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729859    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729861    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729864    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729866    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729869    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729882    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729885    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 02 10:33:18.733772 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729887    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729890    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729892    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729895    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729898    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729900    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729903    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729905    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729908    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729910    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729913    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729915    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729918    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729920    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729923    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729925    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729927    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729930    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729933    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729936    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 02 10:33:18.734322 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729939    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729941    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729943    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729946    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729949    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729951    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729954    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729957    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729960    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729964    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729967    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729969    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729972    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729974    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729977    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729979    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729984    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729987    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729990    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729993    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 02 10:33:18.734804 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729996    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.729998    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730001    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730003    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730005    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730008    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730011    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730013    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730015    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730018    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730020    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730023    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730040    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730043    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730046    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730049    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730052    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730055    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730058    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 02 10:33:18.735299 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730060    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730063    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730066    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730070    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730074    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730077    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730079    2576 feature_gate.go:328] unrecognized feature gate: Example2
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730082    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730085    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730087    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730090    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730093    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730095    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730097    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730100    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730102    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730105    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730107    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730111    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.730114    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 02 10:33:18.735787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730181    2576 flags.go:64] FLAG: --address="0.0.0.0"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730191    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730196    2576 flags.go:64] FLAG: --anonymous-auth="true"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730201    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730204    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730208    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730212    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730216    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730219    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730222    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730226    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730230    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730233    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730236    2576 flags.go:64] FLAG: --cgroup-root=""
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730240    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730243    2576 flags.go:64] FLAG: --client-ca-file=""
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730246    2576 flags.go:64] FLAG: --cloud-provider="external"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730249    2576 flags.go:64] FLAG: --cluster-dns="[]"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730254    2576 flags.go:64] FLAG: --cluster-domain=""
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730257    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730260    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730263    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730266    2576 flags.go:64] FLAG: --container-log-max-files="5"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730270    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 02 10:33:18.736314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730273    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730276    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730279    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730282    2576 flags.go:64] FLAG: --contention-profiling="false"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730285    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730288    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730292    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730295    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730299    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730301    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730304    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730307    2576 flags.go:64] FLAG: --enable-load-reader="false"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730310    2576 flags.go:64] FLAG: --enable-server="true"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730313    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730316    2576 flags.go:64] FLAG: --event-burst="100"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730319    2576 flags.go:64] FLAG: --event-qps="50"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730322    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730325    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730328    2576 flags.go:64] FLAG: --eviction-hard=""
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730332    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730335    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730338    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730342    2576 flags.go:64] FLAG: --eviction-soft=""
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730346    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730349    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 02 10:33:18.736894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730352    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730355    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730358    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730361    2576 flags.go:64] FLAG: --fail-swap-on="true"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730364    2576 flags.go:64] FLAG: --feature-gates=""
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730368    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730371    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730374    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730376    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730379    2576 flags.go:64] FLAG: --healthz-port="10248"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730382    2576 flags.go:64] FLAG: --help="false"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730385    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-135-193.ec2.internal"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730388    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730392    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730395    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730398    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730401    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730404    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730407    2576 flags.go:64] FLAG: --image-service-endpoint=""
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730409    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730412    2576 flags.go:64] FLAG: --kube-api-burst="100"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730415    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730418    2576 flags.go:64] FLAG: --kube-api-qps="50"
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730421    2576 flags.go:64] FLAG: --kube-reserved=""
Aug 02 10:33:18.737552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730423    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730426    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730429    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730432    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730435    2576 flags.go:64] FLAG: --lock-file=""
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730439    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730441    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730445    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730450    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730452    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730455    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730457    2576 flags.go:64] FLAG: --logging-format="text"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730460    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730463    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730466    2576 flags.go:64] FLAG: --manifest-url=""
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730468    2576 flags.go:64] FLAG: --manifest-url-header=""
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730472    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730476    2576 flags.go:64] FLAG: --max-open-files="1000000"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730479    2576 flags.go:64] FLAG: --max-pods="110"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730482    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730485    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730488    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730491    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730494    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730497    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 02 10:33:18.738166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730500    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730506    2576 flags.go:64] FLAG: --node-status-max-images="50"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730509    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730512    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730515    2576 flags.go:64] FLAG: --pod-cidr=""
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730517    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730523    2576 flags.go:64] FLAG: --pod-manifest-path=""
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730526    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730529    2576 flags.go:64] FLAG: --pods-per-core="0"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730532    2576 flags.go:64] FLAG: --port="10250"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730535    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730538    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-048871d38a7e4168c"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730542    2576 flags.go:64] FLAG: --qos-reserved=""
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730546    2576 flags.go:64] FLAG: --read-only-port="10255"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730548    2576 flags.go:64] FLAG: --register-node="true"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730551    2576 flags.go:64] FLAG: --register-with-taints=""
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730554    2576 flags.go:64] FLAG: --registry-burst="10"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730557    2576 flags.go:64] FLAG: --registry-qps="5"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730560    2576 flags.go:64] FLAG: --reserved-cpus=""
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730563    2576 flags.go:64] FLAG: --reserved-memory=""
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730566    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730569    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730572    2576 flags.go:64] FLAG: --rotate-certificates="false"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730574    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730577    2576 flags.go:64] FLAG: --runonce="false"
Aug 02 10:33:18.738782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730580    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730583    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730585    2576 flags.go:64] FLAG: --seccomp-default="false"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730588    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730593    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730596    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730599    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730602    2576 flags.go:64] FLAG: --storage-driver-password="root"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730604    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730607    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730610    2576 flags.go:64] FLAG: --storage-driver-user="root"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730612    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730615    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730618    2576 flags.go:64] FLAG: --system-cgroups=""
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730620    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730625    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730628    2576 flags.go:64] FLAG: --tls-cert-file=""
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730630    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730634    2576 flags.go:64] FLAG: --tls-min-version=""
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730638    2576 flags.go:64] FLAG: --tls-private-key-file=""
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730641    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730644    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730647    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730650    2576 flags.go:64] FLAG: --v="2"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730654    2576 flags.go:64] FLAG: --version="false"
Aug 02 10:33:18.739395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730657    2576 flags.go:64] FLAG: --vmodule=""
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730661    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.730664    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733151    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733166    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733171    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733176    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733180    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733185    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733189    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733193    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733197    2576 feature_gate.go:328] unrecognized feature gate: Example2
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733201    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733206    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733210    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733216    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733222    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733226    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733230    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733234    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 02 10:33:18.739994 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733239    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733278    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733284    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733289    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733293    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733297    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733301    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733305    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733309    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733312    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733315    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733318    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733320    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733323    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733325    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733328    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733331    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733334    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733337    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733339    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 02 10:33:18.740504 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733342    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733344    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733347    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733350    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733352    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733355    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733357    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733360    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733362    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733366    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733370    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733373    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733376    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733381    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733385    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733388    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733391    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733393    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733396    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 02 10:33:18.741017 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733399    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733402    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733404    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733407    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733410    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733413    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733415    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733418    2576 feature_gate.go:328] unrecognized feature gate: Example
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733421    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733423    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733426    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733429    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733431    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733434    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733436    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733439    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733441    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733443    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733446    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733448    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 02 10:33:18.741493 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733451    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733453    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733456    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733458    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733460    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733463    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733466    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733470    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733473    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733476    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733479    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733482    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733484    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733487    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733490    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733492    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733495    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733498    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733501    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 02 10:33:18.741988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733503    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733506    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733509    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.733511    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.733520    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.739785    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.739800    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.739819    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.739828    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739880    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739884    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739888    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739891    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739894    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739897    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 02 10:33:18.742468 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739900    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739904    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739908    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739911    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739914    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739917    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739920    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739923    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739926    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739928    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739931    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739934    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739937    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739939    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739942    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739945    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739948    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739950    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739952    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 02 10:33:18.742829 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739955    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739957    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739960    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739962    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739965    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739968    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739970    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739973    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739977    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739981    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739983    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739986    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739989    2576 feature_gate.go:328] unrecognized feature gate: Example2
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739992    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739994    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739997    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.739999    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740002    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740004    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 02 10:33:18.743297 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740007    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740010    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740012    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740015    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740017    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740020    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740022    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740038    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740041    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740043    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740046    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740048    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740051    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740053    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740056    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740058    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740061    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740064    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740067    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 02 10:33:18.743754 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740070    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740073    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740075    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740078    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740080    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740083    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740085    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740088    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740090    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740093    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740096    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740098    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740101    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740104    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740108    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740111    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740114    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740116    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740119    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740121    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 02 10:33:18.744272 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740124    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740127    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740129    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740131    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740134    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740136    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740139    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740141    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740144    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740146    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740150    2576 feature_gate.go:328] unrecognized feature gate: Example
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740152    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740155    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740158    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740160    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 02 10:33:18.744752 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740163    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.740168    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740272    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740276    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740279    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740283    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740288    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740291    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740293    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740296    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740299    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740302    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740304    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740307    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 02 10:33:18.745231 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740310    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740312    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740315    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740317    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740319    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740322    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740324    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740327    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740329    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740332    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740334    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740337    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740339    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740342    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740345    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740348    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740351    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740353    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740356    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740358    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 02 10:33:18.745574 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740361    2576 feature_gate.go:328] unrecognized feature gate: Example2
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740363    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740366    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740370    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740373    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740376    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740378    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740381    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740384    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740387    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740389    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740392    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740395    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740397    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740400    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740402    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740405    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740407    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740410    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 02 10:33:18.746082 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740412    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740414    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740417    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740419    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740422    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740424    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740426    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740430    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740432    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740435    2576 feature_gate.go:328] unrecognized feature gate: Example
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740438    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740441    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740444    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740446    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740449    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740452    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740454    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740456    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740458    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740461    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 02 10:33:18.746532 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740463    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740466    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740468    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740470    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740472    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740475    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740478    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740480    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740482    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740485    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740487    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740490    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740507    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740511    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740513    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740516    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740519    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740522    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740524    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740527    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 02 10:33:18.747018 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740530    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740533    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740535    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740537    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740540    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740542    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740545    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:18.740547    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.740552    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.740653    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.742404    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.744163    2576 server.go:1019] "Starting client certificate rotation"
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.744254    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 02 10:33:18.747527 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.744312    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 02 10:33:18.747842 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.746306    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 02 10:33:18.747842 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.747457    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 02 10:33:18.759924 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.759906    2576 log.go:25] "Validated CRI v1 runtime API"
Aug 02 10:33:18.762067 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.762050    2576 log.go:25] "Validated CRI v1 image API"
Aug 02 10:33:18.763896 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.763871    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 02 10:33:18.766643 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.766618    2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 90ff915c-b9ef-4a72-8493-a5ca6fdbb1ec:/dev/nvme0n1p4 ab6e05d7-c8e2-4e0b-af8f-868c94f68dfa:/dev/nvme0n1p3]
Aug 02 10:33:18.766698 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.766643    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 02 10:33:18.769947 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.769910    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 02 10:33:18.772696 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.772589    2576 manager.go:216] Machine: {Timestamp:2026-08-02 10:33:18.772125389 +0000 UTC m=+0.302708030 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098016 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2929b9827679fda78ea03263e30ad9 SystemUUID:ec2929b9-8276-79fd-a78e-a03263e30ad9 BootID:7bd732ec-b8ef-4d4d-a42f-902049d2453d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:58:71:b7:1f:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:58:71:b7:1f:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:0e:9b:2d:fb:31 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 02 10:33:18.772696 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.772683    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 02 10:33:18.772817 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.772753    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 02 10:33:18.773682 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.773653    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 02 10:33:18.773815 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.773686    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-193.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 02 10:33:18.774292 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.774272    2576 topology_manager.go:138] "Creating topology manager with none policy"
Aug 02 10:33:18.774292 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.774294    2576 container_manager_linux.go:309] "Creating device plugin manager"
Aug 02 10:33:18.774366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.774302    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 02 10:33:18.774835 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.774826    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 02 10:33:18.775445 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.775435    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 02 10:33:18.779198 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.779185    2576 state_mem.go:36] "Initialized new in-memory state store"
Aug 02 10:33:18.779302 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.779293    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 02 10:33:18.779332 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.779322    2576 kubelet.go:486] "Attempting to sync node with API server"
Aug 02 10:33:18.779332 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.779331    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 02 10:33:18.779394 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.779345    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 02 10:33:18.779394 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.779354    2576 kubelet.go:398] "Adding apiserver pod source"
Aug 02 10:33:18.779394 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.779362    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 02 10:33:18.779754 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.779740    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 02 10:33:18.779815 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.779759    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 02 10:33:18.782615 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.782600    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 02 10:33:18.783755 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.783741    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 02 10:33:18.783803 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.783772    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 02 10:33:18.784703 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784689    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 02 10:33:18.784736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784711    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 02 10:33:18.784736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784718    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 02 10:33:18.784736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784723    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 02 10:33:18.784736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784732    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 02 10:33:18.784736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784738    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 02 10:33:18.784861 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784746    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 02 10:33:18.784861 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784751    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 02 10:33:18.784861 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784758    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 02 10:33:18.784861 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784763    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 02 10:33:18.784861 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784771    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 02 10:33:18.784861 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784776    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 02 10:33:18.784861 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784799    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 02 10:33:18.784861 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.784804    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 02 10:33:18.788835 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.788816    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-193.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 02 10:33:18.788908 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.788867    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 02 10:33:18.789225 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.789204    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 02 10:33:18.789316 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.789238    2576 server.go:1268] "Started kubelet"
Aug 02 10:33:18.789370 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.789284    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 02 10:33:18.789370 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.789324    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 02 10:33:18.789457 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.789408    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 02 10:33:18.790055 ip-10-0-135-193 systemd[1]: Started Kubernetes Kubelet.
Aug 02 10:33:18.790649 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.790633    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 02 10:33:18.792020 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.791988    2576 server.go:310] "Adding debug handlers to kubelet server"
Aug 02 10:33:18.794648 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.793771    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-193.ec2.internal.18c7f67e1d0a1e80  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-193.ec2.internal,UID:ip-10-0-135-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-193.ec2.internal,},FirstTimestamp:2026-08-02 10:33:18.789205632 +0000 UTC m=+0.319788251,LastTimestamp:2026-08-02 10:33:18.789205632 +0000 UTC m=+0.319788251,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-193.ec2.internal,}"
Aug 02 10:33:18.796143 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.794900    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 02 10:33:18.796143 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.795272    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 02 10:33:18.796143 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.795308    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 02 10:33:18.796143 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.795913    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 02 10:33:18.796143 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.795932    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 02 10:33:18.796143 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.796052    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 02 10:33:18.796143 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.796084    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9wcbq"
Aug 02 10:33:18.796479 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.796159    2576 reconstruct.go:97] "Volume reconstruction finished"
Aug 02 10:33:18.796479 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.796167    2576 reconciler.go:29] "Reconciler: start to sync state"
Aug 02 10:33:18.796479 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.796202    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:18.797085 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.797070    2576 factory.go:55] Registering systemd factory
Aug 02 10:33:18.797085 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.797084    2576 factory.go:223] Registration of the systemd container factory successfully
Aug 02 10:33:18.797288 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.797277    2576 factory.go:153] Registering CRI-O factory
Aug 02 10:33:18.797288 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.797289    2576 factory.go:223] Registration of the crio container factory successfully
Aug 02 10:33:18.797360 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.797336    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 02 10:33:18.797394 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.797365    2576 factory.go:103] Registering Raw factory
Aug 02 10:33:18.797394 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.797378    2576 manager.go:1196] Started watching for new ooms in manager
Aug 02 10:33:18.798000 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.797989    2576 manager.go:349] Starting recovery of all containers
Aug 02 10:33:18.801305 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.801271    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 02 10:33:18.801386 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.801336    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 02 10:33:18.806508 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.806483    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9wcbq"
Aug 02 10:33:18.810761 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.810746    2576 manager.go:354] Recovery completed
Aug 02 10:33:18.814992 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.814893    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 02 10:33:18.816625 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.816607    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Aug 02 10:33:18.816718 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.816635    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Aug 02 10:33:18.816718 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.816649    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Aug 02 10:33:18.817192 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.817177    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 02 10:33:18.817192 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.817188    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 02 10:33:18.817263 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.817194    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 02 10:33:18.817263 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.817211    2576 state_mem.go:36] "Initialized new in-memory state store"
Aug 02 10:33:18.817862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.817849    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Aug 02 10:33:18.817915 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.817879    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Aug 02 10:33:18.817915 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.817890    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Aug 02 10:33:18.818435 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.818369    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-193.ec2.internal.18c7f67e1eac7bcb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-193.ec2.internal,UID:ip-10-0-135-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-193.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-193.ec2.internal,},FirstTimestamp:2026-08-02 10:33:18.816623563 +0000 UTC m=+0.347206185,LastTimestamp:2026-08-02 10:33:18.816623563 +0000 UTC m=+0.347206185,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-193.ec2.internal,}"
Aug 02 10:33:18.819083 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.819072    2576 policy_none.go:49] "None policy: Start"
Aug 02 10:33:18.819139 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.819086    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 02 10:33:18.819139 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.819097    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 02 10:33:18.820873 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.820863    2576 policy_none.go:47] "Start"
Aug 02 10:33:18.848633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.848616    2576 manager.go:337] "Starting Device Plugin manager"
Aug 02 10:33:18.848724 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.848649    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 02 10:33:18.848724 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.848663    2576 server.go:87] "Starting device plugin registration server"
Aug 02 10:33:18.848913 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.848901    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 02 10:33:18.848971 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.848916    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 02 10:33:18.849013 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.848984    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 02 10:33:18.849138 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.849106    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 02 10:33:18.849138 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.849129    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 02 10:33:18.849654 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.849636    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 02 10:33:18.849729 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.849678    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:18.942247 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.942200    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 02 10:33:18.944422 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.943623    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 02 10:33:18.944422 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.943653    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 02 10:33:18.944422 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.943677    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 02 10:33:18.944422 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.943711    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 02 10:33:18.946263 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.946247    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 02 10:33:18.949323 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.949308    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 02 10:33:18.950083 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.950062    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Aug 02 10:33:18.950164 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.950093    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Aug 02 10:33:18.950164 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.950105    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Aug 02 10:33:18.950164 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.950126    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-193.ec2.internal"
Aug 02 10:33:18.958428 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:18.958413    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-193.ec2.internal"
Aug 02 10:33:18.958511 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.958436    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-193.ec2.internal\": node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:18.974067 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:18.974003    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:19.044678 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.044637    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"]
Aug 02 10:33:19.044825 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.044739    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 02 10:33:19.046263 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.046246    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Aug 02 10:33:19.046358 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.046281    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Aug 02 10:33:19.046358 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.046294    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Aug 02 10:33:19.048465 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.048450    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 02 10:33:19.048577 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.048563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.048623 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.048592    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 02 10:33:19.049180 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.049164    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Aug 02 10:33:19.049263 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.049193    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Aug 02 10:33:19.049263 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.049205    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Aug 02 10:33:19.049263 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.049208    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Aug 02 10:33:19.049263 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.049230    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Aug 02 10:33:19.049263 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.049246    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Aug 02 10:33:19.051600 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.051585    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.051659 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.051616    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 02 10:33:19.052641 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.052627    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Aug 02 10:33:19.052707 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.052656    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Aug 02 10:33:19.052707 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.052666    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Aug 02 10:33:19.073587 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.073567    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-193.ec2.internal\" not found" node="ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.074221 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.074209    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:19.077785 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.077770    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-193.ec2.internal\" not found" node="ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.097112 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.097090    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/279949cd31c7016bcce4dfa057409684-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"279949cd31c7016bcce4dfa057409684\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.097208 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.097115    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/279949cd31c7016bcce4dfa057409684-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"279949cd31c7016bcce4dfa057409684\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.097208 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.097131    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6d6903f32483272d8f8c0d53924b7252-config\") pod \"kube-apiserver-proxy-ip-10-0-135-193.ec2.internal\" (UID: \"6d6903f32483272d8f8c0d53924b7252\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.174535 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.174510    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:19.198072 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.198023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/279949cd31c7016bcce4dfa057409684-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"279949cd31c7016bcce4dfa057409684\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.198187 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.198081    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/279949cd31c7016bcce4dfa057409684-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"279949cd31c7016bcce4dfa057409684\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.198187 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.198104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/279949cd31c7016bcce4dfa057409684-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"279949cd31c7016bcce4dfa057409684\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.198187 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.198081    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/279949cd31c7016bcce4dfa057409684-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"279949cd31c7016bcce4dfa057409684\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.198187 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.198147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6d6903f32483272d8f8c0d53924b7252-config\") pod \"kube-apiserver-proxy-ip-10-0-135-193.ec2.internal\" (UID: \"6d6903f32483272d8f8c0d53924b7252\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.198187 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.198170    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6d6903f32483272d8f8c0d53924b7252-config\") pod \"kube-apiserver-proxy-ip-10-0-135-193.ec2.internal\" (UID: \"6d6903f32483272d8f8c0d53924b7252\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.275394 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.275324    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:19.375961 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.375934    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:19.376120 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.375985    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.381078 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.380758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.477043 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.476997    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:19.577579 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.577500    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Aug 02 10:33:19.636203 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.636176    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 02 10:33:19.649370 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.649351    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 02 10:33:19.696293 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.696267    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.707415 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.707398    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 02 10:33:19.708966 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.708954    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Aug 02 10:33:19.717562 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.717550    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 02 10:33:19.743711 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.743687    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 02 10:33:19.744144 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.743816    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 02 10:33:19.744144 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.743825    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 02 10:33:19.744144 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.743827    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 02 10:33:19.767361 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.767332    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 02 10:33:19.779533 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.779514    2576 apiserver.go:52] "Watching apiserver"
Aug 02 10:33:19.787746 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.787726    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 02 10:33:19.788721 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.788701    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-7ch8d","kube-system/konnectivity-agent-5l64t","kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg","openshift-cluster-node-tuning-operator/tuned-wz6qg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal","openshift-multus/multus-additional-cni-plugins-2rj9b","openshift-multus/network-metrics-daemon-mmf2n","openshift-ovn-kubernetes/ovnkube-node-j98mk","kube-system/global-pull-secret-syncer-xfdxj","openshift-dns/node-resolver-rkcz8","openshift-image-registry/node-ca-fbkmq","openshift-multus/multus-59tjn","openshift-network-diagnostics/network-check-target-r8c9h"]
Aug 02 10:33:19.791268 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.791252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.793861 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.793844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.794391 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.794359    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 02 10:33:19.794534 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.794497    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 02 10:33:19.794612 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.794591    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 02 10:33:19.794907 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.794892    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 02 10:33:19.794979 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.794917    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 02 10:33:19.794979 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.794961    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9bzd2\""
Aug 02 10:33:19.795570 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.795554    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 02 10:33:19.795840 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.795825    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:19.795913 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.795878    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:19.796148 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.796131    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l4wzk\""
Aug 02 10:33:19.796316 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.796300    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 02 10:33:19.798101 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.798082    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.800336 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.800318    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 02 10:33:19.800429 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.800359    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.800486 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.800435    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 02 10:33:19.800563 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.800542    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v7dmb\""
Aug 02 10:33:19.800563 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.800553    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 02 10:33:19.801332 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-system-cni-dir\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801437 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-os-release\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801437 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-var-lib-cni-multus\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801437 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-registration-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.801437 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2fed233-9b0d-44ad-9778-3ec67dd03656-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.801437 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801416    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c7e82f0-5926-4b13-9dba-ccafb2b13278-cni-binary-copy\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-socket-dir-parent\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-conf-dir\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801480    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-run-multus-certs\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801495    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-cni-dir\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801512    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-cnibin\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801534    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2fed233-9b0d-44ad-9778-3ec67dd03656-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zh4v\" (UniqueName: \"kubernetes.io/projected/a2fed233-9b0d-44ad-9778-3ec67dd03656-kube-api-access-8zh4v\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801570    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-etc-kubernetes\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2fed233-9b0d-44ad-9778-3ec67dd03656-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801609    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801636    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-var-lib-kubelet\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801658    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-hostroot\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-cnibin\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801693    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-os-release\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.801702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801709    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-run-k8s-cni-cncf-io\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801767    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-socket-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801780    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-system-cni-dir\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801793    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-var-lib-cni-bin\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801812    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-daemon-config\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-device-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-run-netns\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801904    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdjvb\" (UniqueName: \"kubernetes.io/projected/5c7e82f0-5926-4b13-9dba-ccafb2b13278-kube-api-access-rdjvb\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801926    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-sys-fs\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.802424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.801968    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw2tz\" (UniqueName: \"kubernetes.io/projected/aba8b894-9906-4627-80ff-62206f14e79c-kube-api-access-fw2tz\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.802927 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.802834    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 02 10:33:19.803071 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.803057    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tpp5r\""
Aug 02 10:33:19.803448 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.803432    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 02 10:33:19.803971 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.803954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:19.804067 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.804041    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:19.806197 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.806182    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.808609 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.808258    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:19.808609 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.808613    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 02 10:33:19.808771 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.808657    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8dx9b\""
Aug 02 10:33:19.808771 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.808661    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 02 10:33:19.809090 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.809076    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 02 10:33:19.810451 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.810437    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:19.812146 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.812115    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-01 10:28:18 +0000 UTC" deadline="2028-01-23 23:03:22.647671521 +0000 UTC"
Aug 02 10:33:19.812146 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.812145    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12948h30m2.835528735s"
Aug 02 10:33:19.812993 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.812979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:19.813087 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.813061    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:19.813401 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.813385    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 02 10:33:19.813534 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.813436    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 02 10:33:19.813733 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.813717    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 02 10:33:19.813817 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.813793    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 02 10:33:19.813928 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.813898    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 02 10:33:19.814186 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.814158    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 02 10:33:19.814870 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.814852    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 02 10:33:19.815211 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.815196    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 02 10:33:19.815423 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.815409    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cvv5t\""
Aug 02 10:33:19.815710 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.815693    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4vhpp\""
Aug 02 10:33:19.815873 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.815856    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:19.815947 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.815881    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 02 10:33:19.818308 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.818285    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:19.818484 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.818466    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 02 10:33:19.819414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.819395    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 02 10:33:19.819491 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.819438    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z5jzw\""
Aug 02 10:33:19.834470 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.834427    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 02 10:33:19.834470 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.834427    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 02 10:33:19.834587 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.834488    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 02 10:33:19.834699 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.834687    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5c7vk\""
Aug 02 10:33:19.858819 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.858804    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rwjnt"
Aug 02 10:33:19.867965 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.867939    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rwjnt"
Aug 02 10:33:19.897119 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.897103    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 02 10:33:19.902466 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2fed233-9b0d-44ad-9778-3ec67dd03656-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.902518 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902480    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-etc-kubernetes\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.902518 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902498    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-sysctl-d\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.902518 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902512    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-942vk\" (UniqueName: \"kubernetes.io/projected/daef7f90-6c0c-4e01-bb1f-b6e25337241e-kube-api-access-942vk\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.902622 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-var-lib-openvswitch\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.902622 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902558    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-etc-kubernetes\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.902622 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:19.902622 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902597    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd96c936-8adc-470d-ba80-3b54d438f858-host-slash\") pod \"iptables-alerter-7ch8d\" (UID: \"dd96c936-8adc-470d-ba80-3b54d438f858\") " pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:19.902622 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwj98\" (UniqueName: \"kubernetes.io/projected/a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786-kube-api-access-vwj98\") pod \"node-resolver-rkcz8\" (UID: \"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786\") " pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:19.902786 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-device-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.902786 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-kubernetes\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.902786 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-kubelet\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.902786 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-host\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.902786 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902742    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-device-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.902786 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-var-lib-kubelet\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902803    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902809    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-var-lib-kubelet\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-run-k8s-cni-cncf-io\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-system-cni-dir\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-sysctl-conf\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902886    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-run-k8s-cni-cncf-io\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902911    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-system-cni-dir\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-run-netns\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.902964    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-run-ovn-kubernetes\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd96c936-8adc-470d-ba80-3b54d438f858-iptables-alerter-script\") pod \"iptables-alerter-7ch8d\" (UID: \"dd96c936-8adc-470d-ba80-3b54d438f858\") " pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903017    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2fed233-9b0d-44ad-9778-3ec67dd03656-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.903069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/92f58dc5-7223-40a2-85bb-020e1ef6d6a1-serviceca\") pod \"node-ca-fbkmq\" (UID: \"92f58dc5-7223-40a2-85bb-020e1ef6d6a1\") " pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdjvb\" (UniqueName: \"kubernetes.io/projected/5c7e82f0-5926-4b13-9dba-ccafb2b13278-kube-api-access-rdjvb\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903101    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw2tz\" (UniqueName: \"kubernetes.io/projected/aba8b894-9906-4627-80ff-62206f14e79c-kube-api-access-fw2tz\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-run\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903135    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-sys\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903156    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903190    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-systemd-units\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903301    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-run-ovn\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903328    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a55c89e-0c43-4f48-be56-546f81bdc357-ovnkube-config\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-system-cni-dir\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-var-lib-cni-multus\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-registration-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-var-lib-cni-multus\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-var-lib-kubelet\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903420    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/daef7f90-6c0c-4e01-bb1f-b6e25337241e-tmp\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903422    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-system-cni-dir\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.903633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903429    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-registration-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903446    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-dbus\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2fed233-9b0d-44ad-9778-3ec67dd03656-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c7e82f0-5926-4b13-9dba-ccafb2b13278-cni-binary-copy\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-socket-dir-parent\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-run-openvswitch\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-log-socket\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a55c89e-0c43-4f48-be56-546f81bdc357-ovnkube-script-lib\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903634    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q66k\" (UniqueName: \"kubernetes.io/projected/4a55c89e-0c43-4f48-be56-546f81bdc357-kube-api-access-7q66k\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-cni-dir\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-socket-dir-parent\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903686    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-cnibin\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zh4v\" (UniqueName: \"kubernetes.io/projected/a2fed233-9b0d-44ad-9778-3ec67dd03656-kube-api-access-8zh4v\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903717    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-sysconfig\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-systemd\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903760    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk6kn\" (UniqueName: \"kubernetes.io/projected/7f208a37-3b97-4072-ac20-9ffbf8732d84-kube-api-access-rk6kn\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-cni-dir\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903783    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786-hosts-file\") pod \"node-resolver-rkcz8\" (UID: \"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786\") " pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903787    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-cnibin\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903803    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92f58dc5-7223-40a2-85bb-020e1ef6d6a1-host\") pod \"node-ca-fbkmq\" (UID: \"92f58dc5-7223-40a2-85bb-020e1ef6d6a1\") " pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2fed233-9b0d-44ad-9778-3ec67dd03656-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-cni-bin\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903834    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2fed233-9b0d-44ad-9778-3ec67dd03656-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903848    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-kubelet-config\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-lib-modules\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903909    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-hostroot\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-cnibin\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-os-release\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-socket-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.903975    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-hostroot\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-cnibin\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904007    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2fed233-9b0d-44ad-9778-3ec67dd03656-os-release\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-var-lib-cni-bin\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.904878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-var-lib-cni-bin\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904092    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-socket-dir\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c7e82f0-5926-4b13-9dba-ccafb2b13278-cni-binary-copy\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-daemon-config\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904127    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-modprobe-d\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4fc2ea2a-24bc-45d0-9953-b121d550ae1a-konnectivity-ca\") pod \"konnectivity-agent-5l64t\" (UID: \"4fc2ea2a-24bc-45d0-9953-b121d550ae1a\") " pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-node-log\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904219    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a55c89e-0c43-4f48-be56-546f81bdc357-ovn-node-metrics-cert\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx7bs\" (UniqueName: \"kubernetes.io/projected/dd96c936-8adc-470d-ba80-3b54d438f858-kube-api-access-lx7bs\") pod \"iptables-alerter-7ch8d\" (UID: \"dd96c936-8adc-470d-ba80-3b54d438f858\") " pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4fc2ea2a-24bc-45d0-9953-b121d550ae1a-agent-certs\") pod \"konnectivity-agent-5l64t\" (UID: \"4fc2ea2a-24bc-45d0-9953-b121d550ae1a\") " pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-run-netns\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2fed233-9b0d-44ad-9778-3ec67dd03656-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-sys-fs\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-slash\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-run-systemd\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-run-netns\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905368 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aba8b894-9906-4627-80ff-62206f14e79c-sys-fs\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904340    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a55c89e-0c43-4f48-be56-546f81bdc357-env-overrides\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904386    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786-tmp-dir\") pod \"node-resolver-rkcz8\" (UID: \"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786\") " pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-os-release\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-tuned\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904466    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j2mc\" (UniqueName: \"kubernetes.io/projected/92f58dc5-7223-40a2-85bb-020e1ef6d6a1-kube-api-access-8j2mc\") pod \"node-ca-fbkmq\" (UID: \"92f58dc5-7223-40a2-85bb-020e1ef6d6a1\") " pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-daemon-config\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-conf-dir\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-os-release\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-run-multus-certs\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-multus-conf-dir\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-etc-openvswitch\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5c7e82f0-5926-4b13-9dba-ccafb2b13278-host-run-multus-certs\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.905787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.904571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-cni-netd\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:19.908825 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.908808    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 02 10:33:19.908825 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.908825    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 02 10:33:19.908923 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.908835    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pfgw5 for pod openshift-network-diagnostics/network-check-target-r8c9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:19.908923 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.908846    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 02 10:33:19.908923 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:19.908883    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5 podName:b2ec253c-efc7-49ee-a2da-5fcd3453c36a nodeName:}" failed. No retries permitted until 2026-08-02 10:33:20.40886774 +0000 UTC m=+1.939450362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pfgw5" (UniqueName: "kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5") pod "network-check-target-r8c9h" (UID: "b2ec253c-efc7-49ee-a2da-5fcd3453c36a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:19.911607 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.911581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdjvb\" (UniqueName: \"kubernetes.io/projected/5c7e82f0-5926-4b13-9dba-ccafb2b13278-kube-api-access-rdjvb\") pod \"multus-59tjn\" (UID: \"5c7e82f0-5926-4b13-9dba-ccafb2b13278\") " pod="openshift-multus/multus-59tjn"
Aug 02 10:33:19.911684 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.911635    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zh4v\" (UniqueName: \"kubernetes.io/projected/a2fed233-9b0d-44ad-9778-3ec67dd03656-kube-api-access-8zh4v\") pod \"multus-additional-cni-plugins-2rj9b\" (UID: \"a2fed233-9b0d-44ad-9778-3ec67dd03656\") " pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:19.911720 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.911706    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw2tz\" (UniqueName: \"kubernetes.io/projected/aba8b894-9906-4627-80ff-62206f14e79c-kube-api-access-fw2tz\") pod \"aws-ebs-csi-driver-node-ccftg\" (UID: \"aba8b894-9906-4627-80ff-62206f14e79c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:19.994448 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:19.994411    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod279949cd31c7016bcce4dfa057409684.slice/crio-c9a7241b8a5a00da7db5032d274537f1642b21674454e2cfc552f13f16635f53 WatchSource:0}: Error finding container c9a7241b8a5a00da7db5032d274537f1642b21674454e2cfc552f13f16635f53: Status 404 returned error can't find the container with id c9a7241b8a5a00da7db5032d274537f1642b21674454e2cfc552f13f16635f53
Aug 02 10:33:19.996225 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:19.996179    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d6903f32483272d8f8c0d53924b7252.slice/crio-7bdfb63a1d719c64e38cba78c290fdd01a9d016840fcb36bffbcf4a5355a8369 WatchSource:0}: Error finding container 7bdfb63a1d719c64e38cba78c290fdd01a9d016840fcb36bffbcf4a5355a8369: Status 404 returned error can't find the container with id 7bdfb63a1d719c64e38cba78c290fdd01a9d016840fcb36bffbcf4a5355a8369
Aug 02 10:33:19.999965 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:19.999950    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 02 10:33:20.005568 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005546    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-run-openvswitch\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.005647 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-log-socket\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.005647 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005588    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a55c89e-0c43-4f48-be56-546f81bdc357-ovnkube-script-lib\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.005647 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q66k\" (UniqueName: \"kubernetes.io/projected/4a55c89e-0c43-4f48-be56-546f81bdc357-kube-api-access-7q66k\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.005647 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-sysconfig\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.005647 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-systemd\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005643    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-log-socket\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk6kn\" (UniqueName: \"kubernetes.io/projected/7f208a37-3b97-4072-ac20-9ffbf8732d84-kube-api-access-rk6kn\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-sysconfig\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786-hosts-file\") pod \"node-resolver-rkcz8\" (UID: \"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786\") " pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-systemd\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92f58dc5-7223-40a2-85bb-020e1ef6d6a1-host\") pod \"node-ca-fbkmq\" (UID: \"92f58dc5-7223-40a2-85bb-020e1ef6d6a1\") " pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-cni-bin\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005775    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786-hosts-file\") pod \"node-resolver-rkcz8\" (UID: \"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786\") " pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-kubelet-config\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005796    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92f58dc5-7223-40a2-85bb-020e1ef6d6a1-host\") pod \"node-ca-fbkmq\" (UID: \"92f58dc5-7223-40a2-85bb-020e1ef6d6a1\") " pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005817    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-cni-bin\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005829    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-run-openvswitch\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-kubelet-config\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-lib-modules\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.005887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-modprobe-d\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4fc2ea2a-24bc-45d0-9953-b121d550ae1a-konnectivity-ca\") pod \"konnectivity-agent-5l64t\" (UID: \"4fc2ea2a-24bc-45d0-9953-b121d550ae1a\") " pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.005984    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-lib-modules\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006008    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-modprobe-d\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.006022    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-node-log\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a55c89e-0c43-4f48-be56-546f81bdc357-ovn-node-metrics-cert\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.006099    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret podName:e17cf15b-0d83-4f4d-afa8-23a6c52838f8 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:20.506084872 +0000 UTC m=+2.036667507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret") pod "global-pull-secret-syncer-xfdxj" (UID: "e17cf15b-0d83-4f4d-afa8-23a6c52838f8") : object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx7bs\" (UniqueName: \"kubernetes.io/projected/dd96c936-8adc-470d-ba80-3b54d438f858-kube-api-access-lx7bs\") pod \"iptables-alerter-7ch8d\" (UID: \"dd96c936-8adc-470d-ba80-3b54d438f858\") " pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006143    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-node-log\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4fc2ea2a-24bc-45d0-9953-b121d550ae1a-agent-certs\") pod \"konnectivity-agent-5l64t\" (UID: \"4fc2ea2a-24bc-45d0-9953-b121d550ae1a\") " pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-slash\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006190    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a55c89e-0c43-4f48-be56-546f81bdc357-ovnkube-script-lib\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-run-systemd\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.006506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006228    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a55c89e-0c43-4f48-be56-546f81bdc357-env-overrides\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006232    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-run-systemd\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006242    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786-tmp-dir\") pod \"node-resolver-rkcz8\" (UID: \"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786\") " pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-tuned\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j2mc\" (UniqueName: \"kubernetes.io/projected/92f58dc5-7223-40a2-85bb-020e1ef6d6a1-kube-api-access-8j2mc\") pod \"node-ca-fbkmq\" (UID: \"92f58dc5-7223-40a2-85bb-020e1ef6d6a1\") " pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-etc-openvswitch\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006310    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-cni-netd\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006323    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-sysctl-d\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-942vk\" (UniqueName: \"kubernetes.io/projected/daef7f90-6c0c-4e01-bb1f-b6e25337241e-kube-api-access-942vk\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-var-lib-openvswitch\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006404    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-etc-openvswitch\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006414    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-slash\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006441    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-cni-netd\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006452    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-var-lib-openvswitch\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006458    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4fc2ea2a-24bc-45d0-9953-b121d550ae1a-konnectivity-ca\") pod \"konnectivity-agent-5l64t\" (UID: \"4fc2ea2a-24bc-45d0-9953-b121d550ae1a\") " pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd96c936-8adc-470d-ba80-3b54d438f858-host-slash\") pod \"iptables-alerter-7ch8d\" (UID: \"dd96c936-8adc-470d-ba80-3b54d438f858\") " pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006505    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd96c936-8adc-470d-ba80-3b54d438f858-host-slash\") pod \"iptables-alerter-7ch8d\" (UID: \"dd96c936-8adc-470d-ba80-3b54d438f858\") " pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-sysctl-d\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.007425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwj98\" (UniqueName: \"kubernetes.io/projected/a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786-kube-api-access-vwj98\") pod \"node-resolver-rkcz8\" (UID: \"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786\") " pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-kubernetes\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006720    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-kubelet\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006743    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-host\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786-tmp-dir\") pod \"node-resolver-rkcz8\" (UID: \"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786\") " pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006768    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-sysctl-conf\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-run-netns\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-run-ovn-kubernetes\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006988    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a55c89e-0c43-4f48-be56-546f81bdc357-env-overrides\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.006994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd96c936-8adc-470d-ba80-3b54d438f858-iptables-alerter-script\") pod \"iptables-alerter-7ch8d\" (UID: \"dd96c936-8adc-470d-ba80-3b54d438f858\") " pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007047    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-sysctl-conf\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/92f58dc5-7223-40a2-85bb-020e1ef6d6a1-serviceca\") pod \"node-ca-fbkmq\" (UID: \"92f58dc5-7223-40a2-85bb-020e1ef6d6a1\") " pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-run-netns\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-kubelet\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007243    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-run\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-sys\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-systemd-units\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.008274 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-run-ovn\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a55c89e-0c43-4f48-be56-546f81bdc357-ovnkube-config\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-sys\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007465    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd96c936-8adc-470d-ba80-3b54d438f858-iptables-alerter-script\") pod \"iptables-alerter-7ch8d\" (UID: \"dd96c936-8adc-470d-ba80-3b54d438f858\") " pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007512    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-kubernetes\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-host\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-run\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007610    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/92f58dc5-7223-40a2-85bb-020e1ef6d6a1-serviceca\") pod \"node-ca-fbkmq\" (UID: \"92f58dc5-7223-40a2-85bb-020e1ef6d6a1\") " pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.007646    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-host-run-ovn-kubernetes\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-systemd-units\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.007746    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:20.507731259 +0000 UTC m=+2.038313877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-var-lib-kubelet\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007817    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a55c89e-0c43-4f48-be56-546f81bdc357-run-ovn\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/daef7f90-6c0c-4e01-bb1f-b6e25337241e-tmp\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007866    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-dbus\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/daef7f90-6c0c-4e01-bb1f-b6e25337241e-var-lib-kubelet\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.007941    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-dbus\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:20.009366 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.008101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a55c89e-0c43-4f48-be56-546f81bdc357-ovnkube-config\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.010094 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.008582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/daef7f90-6c0c-4e01-bb1f-b6e25337241e-etc-tuned\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.010094 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.008671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a55c89e-0c43-4f48-be56-546f81bdc357-ovn-node-metrics-cert\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.010094 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.009105    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4fc2ea2a-24bc-45d0-9953-b121d550ae1a-agent-certs\") pod \"konnectivity-agent-5l64t\" (UID: \"4fc2ea2a-24bc-45d0-9953-b121d550ae1a\") " pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:20.010094 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.009964    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/daef7f90-6c0c-4e01-bb1f-b6e25337241e-tmp\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.014135 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.014114    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx7bs\" (UniqueName: \"kubernetes.io/projected/dd96c936-8adc-470d-ba80-3b54d438f858-kube-api-access-lx7bs\") pod \"iptables-alerter-7ch8d\" (UID: \"dd96c936-8adc-470d-ba80-3b54d438f858\") " pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:20.014466 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.014450    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-942vk\" (UniqueName: \"kubernetes.io/projected/daef7f90-6c0c-4e01-bb1f-b6e25337241e-kube-api-access-942vk\") pod \"tuned-wz6qg\" (UID: \"daef7f90-6c0c-4e01-bb1f-b6e25337241e\") " pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.014658 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.014638    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwj98\" (UniqueName: \"kubernetes.io/projected/a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786-kube-api-access-vwj98\") pod \"node-resolver-rkcz8\" (UID: \"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786\") " pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:20.014707 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.014657    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j2mc\" (UniqueName: \"kubernetes.io/projected/92f58dc5-7223-40a2-85bb-020e1ef6d6a1-kube-api-access-8j2mc\") pod \"node-ca-fbkmq\" (UID: \"92f58dc5-7223-40a2-85bb-020e1ef6d6a1\") " pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:20.014707 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.014698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q66k\" (UniqueName: \"kubernetes.io/projected/4a55c89e-0c43-4f48-be56-546f81bdc357-kube-api-access-7q66k\") pod \"ovnkube-node-j98mk\" (UID: \"4a55c89e-0c43-4f48-be56-546f81bdc357\") " pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.015177 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.015159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk6kn\" (UniqueName: \"kubernetes.io/projected/7f208a37-3b97-4072-ac20-9ffbf8732d84-kube-api-access-rk6kn\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:20.110152 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.110083    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2rj9b"
Aug 02 10:33:20.118844 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:20.118825    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2fed233_9b0d_44ad_9778_3ec67dd03656.slice/crio-7a98a897a226f34544dcd3ba9896c761b0cdb8c4d42b4eff4dc026a747061a25 WatchSource:0}: Error finding container 7a98a897a226f34544dcd3ba9896c761b0cdb8c4d42b4eff4dc026a747061a25: Status 404 returned error can't find the container with id 7a98a897a226f34544dcd3ba9896c761b0cdb8c4d42b4eff4dc026a747061a25
Aug 02 10:33:20.125574 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.125557    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-59tjn"
Aug 02 10:33:20.132756 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:20.132736    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c7e82f0_5926_4b13_9dba_ccafb2b13278.slice/crio-768199d2474e278305892d2d806e85af74f116631b8aeee9b968e32240544826 WatchSource:0}: Error finding container 768199d2474e278305892d2d806e85af74f116631b8aeee9b968e32240544826: Status 404 returned error can't find the container with id 768199d2474e278305892d2d806e85af74f116631b8aeee9b968e32240544826
Aug 02 10:33:20.141314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.141298    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg"
Aug 02 10:33:20.148743 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:20.148725    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba8b894_9906_4627_80ff_62206f14e79c.slice/crio-e1cb159f69875ca3c4399aa21107286ef20ec27322b4a9f60d6c856585148d6a WatchSource:0}: Error finding container e1cb159f69875ca3c4399aa21107286ef20ec27322b4a9f60d6c856585148d6a: Status 404 returned error can't find the container with id e1cb159f69875ca3c4399aa21107286ef20ec27322b4a9f60d6c856585148d6a
Aug 02 10:33:20.151529 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.151514    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wz6qg"
Aug 02 10:33:20.158587 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:20.158566    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaef7f90_6c0c_4e01_bb1f_b6e25337241e.slice/crio-a39bc803d64e51b518234a89acf50c4fc4041d9b105f5589afb96e7206a10bfa WatchSource:0}: Error finding container a39bc803d64e51b518234a89acf50c4fc4041d9b105f5589afb96e7206a10bfa: Status 404 returned error can't find the container with id a39bc803d64e51b518234a89acf50c4fc4041d9b105f5589afb96e7206a10bfa
Aug 02 10:33:20.171859 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.171844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:20.179316 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:20.179294    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a55c89e_0c43_4f48_be56_546f81bdc357.slice/crio-dfb72b24c9e464258ed17f677f10851391f9f37759573af27fcbd3e9dd892195 WatchSource:0}: Error finding container dfb72b24c9e464258ed17f677f10851391f9f37759573af27fcbd3e9dd892195: Status 404 returned error can't find the container with id dfb72b24c9e464258ed17f677f10851391f9f37759573af27fcbd3e9dd892195
Aug 02 10:33:20.185442 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.185423    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ch8d"
Aug 02 10:33:20.192371 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:20.192355    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd96c936_8adc_470d_ba80_3b54d438f858.slice/crio-2a38ac64367e4d9e41012207556f66cbe1c7e2d52f901ffb742d5728ddf9f912 WatchSource:0}: Error finding container 2a38ac64367e4d9e41012207556f66cbe1c7e2d52f901ffb742d5728ddf9f912: Status 404 returned error can't find the container with id 2a38ac64367e4d9e41012207556f66cbe1c7e2d52f901ffb742d5728ddf9f912
Aug 02 10:33:20.212915 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.212896    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:20.220394 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:20.220376    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc2ea2a_24bc_45d0_9953_b121d550ae1a.slice/crio-a636a658a1be579d6933112ed3b2cc968c35872275a0c2c18b27fb9295dc236b WatchSource:0}: Error finding container a636a658a1be579d6933112ed3b2cc968c35872275a0c2c18b27fb9295dc236b: Status 404 returned error can't find the container with id a636a658a1be579d6933112ed3b2cc968c35872275a0c2c18b27fb9295dc236b
Aug 02 10:33:20.221200 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.221184    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkcz8"
Aug 02 10:33:20.228143 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:20.228108    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3cfe13b_9aba_4b1f_8e83_33f2f2d7f786.slice/crio-0eab4c91364d4f5c9cdfdf4e7da8163c41f1a9e9133f1c5c1fb5d5d792c15c77 WatchSource:0}: Error finding container 0eab4c91364d4f5c9cdfdf4e7da8163c41f1a9e9133f1c5c1fb5d5d792c15c77: Status 404 returned error can't find the container with id 0eab4c91364d4f5c9cdfdf4e7da8163c41f1a9e9133f1c5c1fb5d5d792c15c77
Aug 02 10:33:20.237887 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.237871    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fbkmq"
Aug 02 10:33:20.245825 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:20.245809    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92f58dc5_7223_40a2_85bb_020e1ef6d6a1.slice/crio-5fed24acf76a658d2b9ef200f59885f3be70c8710159560e44dd7ce69ce4434b WatchSource:0}: Error finding container 5fed24acf76a658d2b9ef200f59885f3be70c8710159560e44dd7ce69ce4434b: Status 404 returned error can't find the container with id 5fed24acf76a658d2b9ef200f59885f3be70c8710159560e44dd7ce69ce4434b
Aug 02 10:33:20.411105 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.411009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:20.411267 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.411172    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 02 10:33:20.411267 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.411201    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 02 10:33:20.411267 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.411212    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pfgw5 for pod openshift-network-diagnostics/network-check-target-r8c9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:20.411267 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.411267    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5 podName:b2ec253c-efc7-49ee-a2da-5fcd3453c36a nodeName:}" failed. No retries permitted until 2026-08-02 10:33:21.411249648 +0000 UTC m=+2.941832271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfgw5" (UniqueName: "kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5") pod "network-check-target-r8c9h" (UID: "b2ec253c-efc7-49ee-a2da-5fcd3453c36a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:20.512759 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.512162    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:20.512759 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.512233    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:20.512759 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.512344    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:20.512759 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.512395    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:21.512379415 +0000 UTC m=+3.042962026 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:20.512759 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.512451    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:20.512759 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.512479    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret podName:e17cf15b-0d83-4f4d-afa8-23a6c52838f8 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:21.512468295 +0000 UTC m=+3.043050906 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret") pod "global-pull-secret-syncer-xfdxj" (UID: "e17cf15b-0d83-4f4d-afa8-23a6c52838f8") : object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:20.682231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.682153    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 02 10:33:20.871499 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.871352    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-01 10:28:19 +0000 UTC" deadline="2028-01-06 12:47:40.291652374 +0000 UTC"
Aug 02 10:33:20.871499 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.871408    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12530h14m19.420248018s"
Aug 02 10:33:20.945092 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.944144    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:20.945092 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:20.944289    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:20.968329 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.968274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkcz8" event={"ID":"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786","Type":"ContainerStarted","Data":"0eab4c91364d4f5c9cdfdf4e7da8163c41f1a9e9133f1c5c1fb5d5d792c15c77"}
Aug 02 10:33:20.986567 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:20.986534    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg" event={"ID":"aba8b894-9906-4627-80ff-62206f14e79c","Type":"ContainerStarted","Data":"e1cb159f69875ca3c4399aa21107286ef20ec27322b4a9f60d6c856585148d6a"}
Aug 02 10:33:21.003159 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.002253    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rj9b" event={"ID":"a2fed233-9b0d-44ad-9778-3ec67dd03656","Type":"ContainerStarted","Data":"7a98a897a226f34544dcd3ba9896c761b0cdb8c4d42b4eff4dc026a747061a25"}
Aug 02 10:33:21.006115 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.004903    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fbkmq" event={"ID":"92f58dc5-7223-40a2-85bb-020e1ef6d6a1","Type":"ContainerStarted","Data":"5fed24acf76a658d2b9ef200f59885f3be70c8710159560e44dd7ce69ce4434b"}
Aug 02 10:33:21.006228 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.006150    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5l64t" event={"ID":"4fc2ea2a-24bc-45d0-9953-b121d550ae1a","Type":"ContainerStarted","Data":"a636a658a1be579d6933112ed3b2cc968c35872275a0c2c18b27fb9295dc236b"}
Aug 02 10:33:21.029215 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.029154    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ch8d" event={"ID":"dd96c936-8adc-470d-ba80-3b54d438f858","Type":"ContainerStarted","Data":"2a38ac64367e4d9e41012207556f66cbe1c7e2d52f901ffb742d5728ddf9f912"}
Aug 02 10:33:21.051742 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.051688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerStarted","Data":"dfb72b24c9e464258ed17f677f10851391f9f37759573af27fcbd3e9dd892195"}
Aug 02 10:33:21.057790 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.057763    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wz6qg" event={"ID":"daef7f90-6c0c-4e01-bb1f-b6e25337241e","Type":"ContainerStarted","Data":"a39bc803d64e51b518234a89acf50c4fc4041d9b105f5589afb96e7206a10bfa"}
Aug 02 10:33:21.063357 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.063327    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-59tjn" event={"ID":"5c7e82f0-5926-4b13-9dba-ccafb2b13278","Type":"ContainerStarted","Data":"768199d2474e278305892d2d806e85af74f116631b8aeee9b968e32240544826"}
Aug 02 10:33:21.073697 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.073670    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal" event={"ID":"6d6903f32483272d8f8c0d53924b7252","Type":"ContainerStarted","Data":"7bdfb63a1d719c64e38cba78c290fdd01a9d016840fcb36bffbcf4a5355a8369"}
Aug 02 10:33:21.081171 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.081144    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal" event={"ID":"279949cd31c7016bcce4dfa057409684","Type":"ContainerStarted","Data":"c9a7241b8a5a00da7db5032d274537f1642b21674454e2cfc552f13f16635f53"}
Aug 02 10:33:21.420800 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.420763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:21.421008 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.420972    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 02 10:33:21.421008 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.421000    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 02 10:33:21.421157 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.421013    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pfgw5 for pod openshift-network-diagnostics/network-check-target-r8c9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:21.421157 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.421087    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5 podName:b2ec253c-efc7-49ee-a2da-5fcd3453c36a nodeName:}" failed. No retries permitted until 2026-08-02 10:33:23.421068027 +0000 UTC m=+4.951650636 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfgw5" (UniqueName: "kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5") pod "network-check-target-r8c9h" (UID: "b2ec253c-efc7-49ee-a2da-5fcd3453c36a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:21.521602 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.521550    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:21.521759 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.521616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:21.521813 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.521762    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:21.521881 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.521819    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret podName:e17cf15b-0d83-4f4d-afa8-23a6c52838f8 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:23.521802934 +0000 UTC m=+5.052385546 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret") pod "global-pull-secret-syncer-xfdxj" (UID: "e17cf15b-0d83-4f4d-afa8-23a6c52838f8") : object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:21.522232 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.522212    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:21.522324 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.522268    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:23.522252566 +0000 UTC m=+5.052835177 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:21.636868 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.636841    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 02 10:33:21.872335 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.872253    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-01 10:28:19 +0000 UTC" deadline="2028-02-03 12:50:14.08409021 +0000 UTC"
Aug 02 10:33:21.872335 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.872291    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13202h16m52.211803165s"
Aug 02 10:33:21.944714 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.944101    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:21.944714 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.944223    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:21.944714 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:21.944572    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:21.944714 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:21.944666    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:22.343889 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:22.343857    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 02 10:33:22.946451 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:22.946426    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:22.946868 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:22.946516    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:23.438223 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:23.438185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:23.438403 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.438390    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 02 10:33:23.438447 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.438410    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 02 10:33:23.438447 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.438422    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pfgw5 for pod openshift-network-diagnostics/network-check-target-r8c9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:23.438553 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.438471    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5 podName:b2ec253c-efc7-49ee-a2da-5fcd3453c36a nodeName:}" failed. No retries permitted until 2026-08-02 10:33:27.438456195 +0000 UTC m=+8.969038804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfgw5" (UniqueName: "kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5") pod "network-check-target-r8c9h" (UID: "b2ec253c-efc7-49ee-a2da-5fcd3453c36a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:23.539287 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:23.539244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:23.539462 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:23.539310    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:23.539462 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.539395    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:23.539462 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.539407    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:23.539462 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.539455    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:27.539438051 +0000 UTC m=+9.070020669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:23.539685 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.539470    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret podName:e17cf15b-0d83-4f4d-afa8-23a6c52838f8 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:27.539462388 +0000 UTC m=+9.070044994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret") pod "global-pull-secret-syncer-xfdxj" (UID: "e17cf15b-0d83-4f4d-afa8-23a6c52838f8") : object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:23.944866 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:23.944834    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:23.945068 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.944975    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:23.945353 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:23.945334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:23.945444 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:23.945426    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:24.946543 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:24.946508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:24.946977 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:24.946688    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:25.943900 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:25.943872    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:25.944104 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:25.943995    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:25.944171 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:25.944107    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:25.944216 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:25.944194    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:26.944101 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:26.944065    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:26.944562 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:26.944193    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:27.471921 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:27.471882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:27.472126 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.472054    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 02 10:33:27.472126 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.472078    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 02 10:33:27.472126 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.472090    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pfgw5 for pod openshift-network-diagnostics/network-check-target-r8c9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:27.472301 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.472149    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5 podName:b2ec253c-efc7-49ee-a2da-5fcd3453c36a nodeName:}" failed. No retries permitted until 2026-08-02 10:33:35.472129509 +0000 UTC m=+17.002712133 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfgw5" (UniqueName: "kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5") pod "network-check-target-r8c9h" (UID: "b2ec253c-efc7-49ee-a2da-5fcd3453c36a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:27.573043 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:27.572991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:27.573220 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:27.573075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:27.573280 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.573220    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:27.573280 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.573277    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret podName:e17cf15b-0d83-4f4d-afa8-23a6c52838f8 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:35.573259323 +0000 UTC m=+17.103841934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret") pod "global-pull-secret-syncer-xfdxj" (UID: "e17cf15b-0d83-4f4d-afa8-23a6c52838f8") : object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:27.573664 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.573639    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:27.573804 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.573693    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:35.573676434 +0000 UTC m=+17.104259046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:27.944809 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:27.944219    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:27.944809 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.944361    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:27.944809 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:27.944225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:27.944809 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:27.944763    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:28.944954 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:28.944897    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:28.945438 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:28.945022    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:29.944199 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:29.944167    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:29.944372 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:29.944288    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:29.944372 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:29.944323    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:29.944497 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:29.944404    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:30.946908 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:30.946876    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:30.947373 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:30.946980    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:31.944297 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:31.943857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:31.944297 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:31.943980    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:31.944297 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:31.944016    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:31.944297 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:31.944130    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:32.944394 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:32.944361    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:32.944813 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:32.944493    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:33.943941 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:33.943902    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:33.944155 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:33.943902    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:33.944155 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:33.944062    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:33.944155 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:33.944138    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:34.944087 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:34.944050    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:34.944524 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:34.944179    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:35.534265 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:35.534230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:35.534449 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.534387    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 02 10:33:35.534449 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.534409    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 02 10:33:35.534449 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.534420    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pfgw5 for pod openshift-network-diagnostics/network-check-target-r8c9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:35.534586 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.534478    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5 podName:b2ec253c-efc7-49ee-a2da-5fcd3453c36a nodeName:}" failed. No retries permitted until 2026-08-02 10:33:51.534463476 +0000 UTC m=+33.065046083 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfgw5" (UniqueName: "kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5") pod "network-check-target-r8c9h" (UID: "b2ec253c-efc7-49ee-a2da-5fcd3453c36a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:35.634814 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:35.634781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:35.634989 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:35.634840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:35.634989 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.634914    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:35.634989 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.634930    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:35.634989 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.634985    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret podName:e17cf15b-0d83-4f4d-afa8-23a6c52838f8 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:51.634971971 +0000 UTC m=+33.165554592 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret") pod "global-pull-secret-syncer-xfdxj" (UID: "e17cf15b-0d83-4f4d-afa8-23a6c52838f8") : object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:35.635199 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.635005    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:51.634995145 +0000 UTC m=+33.165577753 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:35.944155 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:35.944122    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:35.944653 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:35.944122    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:35.944653 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.944248    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:35.944653 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:35.944350    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:36.944878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:36.944417    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:36.944878 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:36.944525    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:37.944418 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:37.944387    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:37.944418 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:37.944413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:37.944631 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:37.944490    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:37.944681 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:37.944627    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:38.944589 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:38.944556    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:38.944997 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:38.944647    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:39.944576 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:39.944447    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:39.944686 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:39.944531    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:39.944686 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:39.944653    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:39.945285 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:39.944734    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:40.128938 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:40.128842    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wz6qg" event={"ID":"daef7f90-6c0c-4e01-bb1f-b6e25337241e","Type":"ContainerStarted","Data":"6efd5a6e4c223975c04745b1564d5a466d42fc5f16e450a14c787ba4991cda96"}
Aug 02 10:33:40.130836 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:40.130814    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-59tjn" event={"ID":"5c7e82f0-5926-4b13-9dba-ccafb2b13278","Type":"ContainerStarted","Data":"1098da34f7d1e790957cc6e83f5a274f573b2e5ea126c03edd16d3d243245f72"}
Aug 02 10:33:40.132350 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:40.132326    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal" event={"ID":"6d6903f32483272d8f8c0d53924b7252","Type":"ContainerStarted","Data":"ab762534dddcc12323c5fed2377cf107f3f938e80e5fba98fad6172f9adde192"}
Aug 02 10:33:40.146565 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:40.146527    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wz6qg" podStartSLOduration=2.639617058 podStartE2EDuration="22.146514685s" totalImagesPullingTime="19.506897627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:40.146236268 +0000 UTC m=+21.676818897" watchObservedRunningTime="2026-08-02 10:33:40.146514685 +0000 UTC m=+21.677097313"
Aug 02 10:33:40.160704 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:40.160662    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal" podStartSLOduration=21.160645913 podStartE2EDuration="21.160645913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:40.159817906 +0000 UTC m=+21.690400554" watchObservedRunningTime="2026-08-02 10:33:40.160645913 +0000 UTC m=+21.691228540"
Aug 02 10:33:40.944740 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:40.944555    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:40.945207 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:40.944845    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:41.135723 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.135692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fbkmq" event={"ID":"92f58dc5-7223-40a2-85bb-020e1ef6d6a1","Type":"ContainerStarted","Data":"6a74b6810402d1f469dc40c8220b7be4c968641a45a51f07e40c441713199b07"}
Aug 02 10:33:41.139844 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.139815    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5l64t" event={"ID":"4fc2ea2a-24bc-45d0-9953-b121d550ae1a","Type":"ContainerStarted","Data":"67ecda06d3478e0858f3908a32eefb167c1de381fcbbcdf5cdab97afc8dfe890"}
Aug 02 10:33:41.141261 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.141226    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ch8d" event={"ID":"dd96c936-8adc-470d-ba80-3b54d438f858","Type":"ContainerStarted","Data":"bac819fbd7db18782971a2fcc0e98c5282b9a2584ef5e8f3de3072c6b32bae41"}
Aug 02 10:33:41.143654 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.143638    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:33:41.143915 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.143898    2576 generic.go:356] "Generic (PLEG): container finished" podID="4a55c89e-0c43-4f48-be56-546f81bdc357" containerID="365bdf6a96eb74eaf4e0bb48cf28791c1a387833e3bbf485902e897791d681bb" exitCode=1
Aug 02 10:33:41.143975 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.143960    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerStarted","Data":"6961d7c95cceff67fd4523fbdcd899cba332430f8cd8541570067080be446e64"}
Aug 02 10:33:41.144019 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.144012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerStarted","Data":"5723cc57e4342dfd33d308d485cbcd41acbe6e126585934c4142777ce3c8c6c3"}
Aug 02 10:33:41.144091 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.144062    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerStarted","Data":"59fe33daeaec32feefc06ee51c4c340422cbf6f357c9281aa975b92a344f441d"}
Aug 02 10:33:41.144130 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.144098    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerStarted","Data":"267f68760705fc43a0c1d583a4566672a8ed31ea943caf7bc73da5ce835a1550"}
Aug 02 10:33:41.144171 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.144132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerDied","Data":"365bdf6a96eb74eaf4e0bb48cf28791c1a387833e3bbf485902e897791d681bb"}
Aug 02 10:33:41.144218 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.144176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerStarted","Data":"4b51286949f3068b5cb4a53594b408f1afda5c09260204ea0c63c93d120b847c"}
Aug 02 10:33:41.145316 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.145273    2576 generic.go:356] "Generic (PLEG): container finished" podID="279949cd31c7016bcce4dfa057409684" containerID="ce578770bf171873099d8ab3421bb322f6fb4a643ddc7b26b0d1e2f340f4364e" exitCode=0
Aug 02 10:33:41.145371 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.145313    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal" event={"ID":"279949cd31c7016bcce4dfa057409684","Type":"ContainerDied","Data":"ce578770bf171873099d8ab3421bb322f6fb4a643ddc7b26b0d1e2f340f4364e"}
Aug 02 10:33:41.148373 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.148353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkcz8" event={"ID":"a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786","Type":"ContainerStarted","Data":"aa4e4191eb0db03f91edea8440b89c7438648c92186abe8ebe76e2dd0ce5d508"}
Aug 02 10:33:41.149635 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.149616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg" event={"ID":"aba8b894-9906-4627-80ff-62206f14e79c","Type":"ContainerStarted","Data":"7ca7fc0f6a4edb700ebcc85512bd89e29c66b5e1e88c3663541b5f6e3e348b30"}
Aug 02 10:33:41.150756 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.150712    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fbkmq" podStartSLOduration=3.7524189039999998 podStartE2EDuration="23.15069561s" totalImagesPullingTime="19.398276706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:41.150130941 +0000 UTC m=+22.680713570" watchObservedRunningTime="2026-08-02 10:33:41.15069561 +0000 UTC m=+22.681278240"
Aug 02 10:33:41.150844 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.150788    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-59tjn" podStartSLOduration=3.615282489 podStartE2EDuration="23.150782688s" totalImagesPullingTime="19.535500199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:40.177801196 +0000 UTC m=+21.708383826" watchObservedRunningTime="2026-08-02 10:33:41.150782688 +0000 UTC m=+22.681365318"
Aug 02 10:33:41.150999 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.150983    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2fed233-9b0d-44ad-9778-3ec67dd03656" containerID="8100d776830f2a2947abd930a4989198e4afb39a84f0538557877f618dc0bf1d" exitCode=0
Aug 02 10:33:41.151087 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.151073    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rj9b" event={"ID":"a2fed233-9b0d-44ad-9778-3ec67dd03656","Type":"ContainerDied","Data":"8100d776830f2a2947abd930a4989198e4afb39a84f0538557877f618dc0bf1d"}
Aug 02 10:33:41.163688 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.163653    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7ch8d" podStartSLOduration=3.74232516 podStartE2EDuration="23.163637349s" totalImagesPullingTime="19.421312189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:41.16326268 +0000 UTC m=+22.693845306" watchObservedRunningTime="2026-08-02 10:33:41.163637349 +0000 UTC m=+22.694219978"
Aug 02 10:33:41.177704 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.177519    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rkcz8" podStartSLOduration=3.738388966 podStartE2EDuration="23.17750676s" totalImagesPullingTime="19.439117794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:41.177438138 +0000 UTC m=+22.708020792" watchObservedRunningTime="2026-08-02 10:33:41.17750676 +0000 UTC m=+22.708089390"
Aug 02 10:33:41.205166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.205125    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5l64t" podStartSLOduration=3.760378154 podStartE2EDuration="23.205109584s" totalImagesPullingTime="19.44473143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:41.204778251 +0000 UTC m=+22.735360880" watchObservedRunningTime="2026-08-02 10:33:41.205109584 +0000 UTC m=+22.735692216"
Aug 02 10:33:41.944443 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.944405    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:41.944643 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:41.944541    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:41.944643 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:41.944600    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:41.944744 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:41.944712    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:42.155130 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:42.155081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal" event={"ID":"279949cd31c7016bcce4dfa057409684","Type":"ContainerStarted","Data":"742b73735734e3daae0a2dcdb477cd5b84a905b330398c1c701251e65a7b2ebe"}
Aug 02 10:33:42.169925 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:42.169874    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal" podStartSLOduration=23.16985496 podStartE2EDuration="23.16985496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:42.169440407 +0000 UTC m=+23.700023037" watchObservedRunningTime="2026-08-02 10:33:42.16985496 +0000 UTC m=+23.700437589"
Aug 02 10:33:42.316425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:42.316395    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 02 10:33:42.868793 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:42.868658    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-02T10:33:42.316412198Z","UUID":"ff39638f-2a4d-4a11-a65e-1092717eea34","Handler":null,"Name":"","Endpoint":""}
Aug 02 10:33:42.870337 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:42.870313    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 02 10:33:42.870482 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:42.870357    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 02 10:33:42.944819 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:42.944798    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:42.944927 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:42.944897    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:43.161189 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:43.161159    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:33:43.161724 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:43.161699    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerStarted","Data":"0e233b9911d56a92c79aae5aceb75c670791f0c0c33d5e9dc01d238dfeb650de"}
Aug 02 10:33:43.163752 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:43.163722    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg" event={"ID":"aba8b894-9906-4627-80ff-62206f14e79c","Type":"ContainerStarted","Data":"8e064ba6af5e40ac9b6147b85bece71613da6e19a42cb01f403711ffef745cb7"}
Aug 02 10:33:43.944487 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:43.944452    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:43.944659 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:43.944495    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:43.944659 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:43.944583    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:43.944765 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:43.944722    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:44.115257 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:44.115221    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:44.116048 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:44.116008    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:44.167961 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:44.167927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg" event={"ID":"aba8b894-9906-4627-80ff-62206f14e79c","Type":"ContainerStarted","Data":"f294ced3ff53188c62050b09be211bc7bdeeb8f876d4f579d18fe9d7ca5642be"}
Aug 02 10:33:44.168421 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:44.168140    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:44.168653 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:44.168631    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5l64t"
Aug 02 10:33:44.186022 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:44.185971    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ccftg" podStartSLOduration=4.311623011 podStartE2EDuration="26.185957027s" totalImagesPullingTime="21.874334016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:44.185214571 +0000 UTC m=+25.715797223" watchObservedRunningTime="2026-08-02 10:33:44.185957027 +0000 UTC m=+25.716539656"
Aug 02 10:33:44.944776 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:44.944738    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:44.944938 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:44.944871    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:45.944745 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:45.944586    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:45.945257 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:45.944603    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:45.945257 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:45.944832    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:45.945257 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:45.944894    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:46.174816 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.174790    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:33:46.175211 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.175189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerStarted","Data":"8c9b6367b98be4ca9309a2ac9e296e3b7bb22a1876139b25cf6497b8e9fcad6d"}
Aug 02 10:33:46.175534 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.175517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:46.175611 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.175600    2576 scope.go:117] "RemoveContainer" containerID="365bdf6a96eb74eaf4e0bb48cf28791c1a387833e3bbf485902e897791d681bb"
Aug 02 10:33:46.177112 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.177090    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2fed233-9b0d-44ad-9778-3ec67dd03656" containerID="c76834a536481543fd584a135f49ca6037d3a7fe088c94985001b61b62448f86" exitCode=0
Aug 02 10:33:46.177225 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.177137    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rj9b" event={"ID":"a2fed233-9b0d-44ad-9778-3ec67dd03656","Type":"ContainerDied","Data":"c76834a536481543fd584a135f49ca6037d3a7fe088c94985001b61b62448f86"}
Aug 02 10:33:46.191894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.191875    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:46.333806 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.333778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5l64t_4fc2ea2a-24bc-45d0-9953-b121d550ae1a/konnectivity-agent/0.log" limit=1024
Aug 02 10:33:46.425612 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.425533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5l64t_4fc2ea2a-24bc-45d0-9953-b121d550ae1a/konnectivity-agent/0.log" limit=1024
Aug 02 10:33:46.944626 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:46.944599    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:46.944789 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:46.944701    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:47.182814 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.182789    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:33:47.183111 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.183088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" event={"ID":"4a55c89e-0c43-4f48-be56-546f81bdc357","Type":"ContainerStarted","Data":"ee56cc66503c39cddd7387ef984a2674ef171058a012711a0b9c300b4292a5e7"}
Aug 02 10:33:47.183329 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.183311    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:47.183399 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.183387    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:47.199795 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.199773    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:33:47.217646 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.217606    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk" podStartSLOduration=9.22798169 podStartE2EDuration="29.217592049s" totalImagesPullingTime="19.989610359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:47.215770978 +0000 UTC m=+28.746353618" watchObservedRunningTime="2026-08-02 10:33:47.217592049 +0000 UTC m=+28.748174677"
Aug 02 10:33:47.880039 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.879840    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mmf2n"]
Aug 02 10:33:47.880203 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.880137    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:47.880267 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:47.880244    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:47.880719 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.880683    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r8c9h"]
Aug 02 10:33:47.880820 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.880781    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:47.880892 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:47.880871    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:47.881339 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.881316    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xfdxj"]
Aug 02 10:33:47.881441 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:47.881399    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:47.881499 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:47.881468    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:48.187600 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:48.187568    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2fed233-9b0d-44ad-9778-3ec67dd03656" containerID="33b18796c9ec99392b253dbcf925b89e29075febaf1b3fcc18e98958ccb2136f" exitCode=0
Aug 02 10:33:48.188107 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:48.187658    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rj9b" event={"ID":"a2fed233-9b0d-44ad-9778-3ec67dd03656","Type":"ContainerDied","Data":"33b18796c9ec99392b253dbcf925b89e29075febaf1b3fcc18e98958ccb2136f"}
Aug 02 10:33:49.944796 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:49.944770    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:49.945136 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:49.944799    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:49.945136 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:49.944801    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:49.945136 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:49.944853    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:49.945136 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:49.944958    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:49.945136 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:49.945073    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:50.194144 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:50.194107    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2fed233-9b0d-44ad-9778-3ec67dd03656" containerID="d285302fcdafe71cab917cdf11aa01137095fe6a23f5e899e2b7a5ba2e69fc4c" exitCode=0
Aug 02 10:33:50.194295 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:50.194170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rj9b" event={"ID":"a2fed233-9b0d-44ad-9778-3ec67dd03656","Type":"ContainerDied","Data":"d285302fcdafe71cab917cdf11aa01137095fe6a23f5e899e2b7a5ba2e69fc4c"}
Aug 02 10:33:51.552989 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:51.552904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:51.553424 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.553100    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 02 10:33:51.553424 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.553121    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 02 10:33:51.553424 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.553132    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pfgw5 for pod openshift-network-diagnostics/network-check-target-r8c9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:51.553424 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.553190    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5 podName:b2ec253c-efc7-49ee-a2da-5fcd3453c36a nodeName:}" failed. No retries permitted until 2026-08-02 10:34:23.553174688 +0000 UTC m=+65.083757296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfgw5" (UniqueName: "kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5") pod "network-check-target-r8c9h" (UID: "b2ec253c-efc7-49ee-a2da-5fcd3453c36a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 02 10:33:51.654067 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:51.654019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:51.654231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:51.654100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:51.654231 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.654182    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:51.654231 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.654185    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:51.654400 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.654240    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:34:23.654223172 +0000 UTC m=+65.184805779 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 02 10:33:51.654400 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.654261    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret podName:e17cf15b-0d83-4f4d-afa8-23a6c52838f8 nodeName:}" failed. No retries permitted until 2026-08-02 10:34:23.654249909 +0000 UTC m=+65.184832516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret") pod "global-pull-secret-syncer-xfdxj" (UID: "e17cf15b-0d83-4f4d-afa8-23a6c52838f8") : object "kube-system"/"original-pull-secret" not registered
Aug 02 10:33:51.944867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:51.944832    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:51.945123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:51.944949    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:51.945123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:51.944965    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:51.945123 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.944961    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfdxj" podUID="e17cf15b-0d83-4f4d-afa8-23a6c52838f8"
Aug 02 10:33:51.945123 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.945018    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8c9h" podUID="b2ec253c-efc7-49ee-a2da-5fcd3453c36a"
Aug 02 10:33:51.945359 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:51.945131    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:33:52.781957 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.781714    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeReady"
Aug 02 10:33:52.782559 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.782074    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 02 10:33:52.824728 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.824697    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6hstf"]
Aug 02 10:33:52.828547 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.828523    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2t452"]
Aug 02 10:33:52.828683 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.828648    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:52.831344 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.831326    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 02 10:33:52.831443 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.831412    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nv7kc\""
Aug 02 10:33:52.831443 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.831438    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 02 10:33:52.832339 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.832321    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2t452"
Aug 02 10:33:52.835219 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.835199    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 02 10:33:52.835219 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.835211    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 02 10:33:52.835464 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.835308    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wbhqw\""
Aug 02 10:33:52.835464 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.835357    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 02 10:33:52.841290 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.841271    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6hstf"]
Aug 02 10:33:52.841778 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.841755    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2t452"]
Aug 02 10:33:52.962777 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.962740    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s68rg\" (UniqueName: \"kubernetes.io/projected/9cab4866-7565-40c3-bbc3-c3c7d2e9656a-kube-api-access-s68rg\") pod \"ingress-canary-2t452\" (UID: \"9cab4866-7565-40c3-bbc3-c3c7d2e9656a\") " pod="openshift-ingress-canary/ingress-canary-2t452"
Aug 02 10:33:52.962918 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.962789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9cab4866-7565-40c3-bbc3-c3c7d2e9656a-cert\") pod \"ingress-canary-2t452\" (UID: \"9cab4866-7565-40c3-bbc3-c3c7d2e9656a\") " pod="openshift-ingress-canary/ingress-canary-2t452"
Aug 02 10:33:52.962918 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.962843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plw5p\" (UniqueName: \"kubernetes.io/projected/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-kube-api-access-plw5p\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:52.962918 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.962896    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:52.963069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.962922    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-tmp-dir\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:52.963069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:52.962999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-config-volume\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:53.063607 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.063521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:53.063607 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.063563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-tmp-dir\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:53.063607 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.063595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-config-volume\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:53.063869 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.063632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s68rg\" (UniqueName: \"kubernetes.io/projected/9cab4866-7565-40c3-bbc3-c3c7d2e9656a-kube-api-access-s68rg\") pod \"ingress-canary-2t452\" (UID: \"9cab4866-7565-40c3-bbc3-c3c7d2e9656a\") " pod="openshift-ingress-canary/ingress-canary-2t452"
Aug 02 10:33:53.063869 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.063660    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9cab4866-7565-40c3-bbc3-c3c7d2e9656a-cert\") pod \"ingress-canary-2t452\" (UID: \"9cab4866-7565-40c3-bbc3-c3c7d2e9656a\") " pod="openshift-ingress-canary/ingress-canary-2t452"
Aug 02 10:33:53.063869 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.063687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plw5p\" (UniqueName: \"kubernetes.io/projected/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-kube-api-access-plw5p\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:53.063869 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:53.063708    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 02 10:33:53.063869 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:53.063777    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls podName:0d7a5405-d4e4-40b8-90c9-06f6b9160b12 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:53.563757009 +0000 UTC m=+35.094339629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls") pod "dns-default-6hstf" (UID: "0d7a5405-d4e4-40b8-90c9-06f6b9160b12") : secret "dns-default-metrics-tls" not found
Aug 02 10:33:53.064148 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.063975    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-tmp-dir\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:53.064197 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.064180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-config-volume\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:53.068631 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.068606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9cab4866-7565-40c3-bbc3-c3c7d2e9656a-cert\") pod \"ingress-canary-2t452\" (UID: \"9cab4866-7565-40c3-bbc3-c3c7d2e9656a\") " pod="openshift-ingress-canary/ingress-canary-2t452"
Aug 02 10:33:53.072469 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.072445    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plw5p\" (UniqueName: \"kubernetes.io/projected/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-kube-api-access-plw5p\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:53.072735 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.072711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s68rg\" (UniqueName: \"kubernetes.io/projected/9cab4866-7565-40c3-bbc3-c3c7d2e9656a-kube-api-access-s68rg\") pod \"ingress-canary-2t452\" (UID: \"9cab4866-7565-40c3-bbc3-c3c7d2e9656a\") " pod="openshift-ingress-canary/ingress-canary-2t452"
Aug 02 10:33:53.144269 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.144239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2t452"
Aug 02 10:33:53.568009 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.567974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:53.568206 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:53.568143    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 02 10:33:53.568261 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:53.568220    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls podName:0d7a5405-d4e4-40b8-90c9-06f6b9160b12 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:54.568199994 +0000 UTC m=+36.098782605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls") pod "dns-default-6hstf" (UID: "0d7a5405-d4e4-40b8-90c9-06f6b9160b12") : secret "dns-default-metrics-tls" not found
Aug 02 10:33:53.944828 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.944796    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:33:53.945316 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.944935    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:33:53.945316 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.944801    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:33:53.949564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.949139    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 02 10:33:53.949564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.949207    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 02 10:33:53.949564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.949218    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dvcgg\""
Aug 02 10:33:53.949564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.949342    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 02 10:33:53.949564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.949419    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 02 10:33:53.949564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:53.949504    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q9bgf\""
Aug 02 10:33:54.576437 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:54.576401    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:54.576609 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:54.576530    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 02 10:33:54.576609 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:54.576587    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls podName:0d7a5405-d4e4-40b8-90c9-06f6b9160b12 nodeName:}" failed. No retries permitted until 2026-08-02 10:33:56.576572881 +0000 UTC m=+38.107155488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls") pod "dns-default-6hstf" (UID: "0d7a5405-d4e4-40b8-90c9-06f6b9160b12") : secret "dns-default-metrics-tls" not found
Aug 02 10:33:54.777810 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:54.777752    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2t452"]
Aug 02 10:33:54.781019 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:33:54.780998    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cab4866_7565_40c3_bbc3_c3c7d2e9656a.slice/crio-fe8d2b0b13fcd2bc72dfb05130ed5f657a32c7250962b2dd1c40fc1132d5f198 WatchSource:0}: Error finding container fe8d2b0b13fcd2bc72dfb05130ed5f657a32c7250962b2dd1c40fc1132d5f198: Status 404 returned error can't find the container with id fe8d2b0b13fcd2bc72dfb05130ed5f657a32c7250962b2dd1c40fc1132d5f198
Aug 02 10:33:55.209099 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:55.208995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2t452" event={"ID":"9cab4866-7565-40c3-bbc3-c3c7d2e9656a","Type":"ContainerStarted","Data":"fe8d2b0b13fcd2bc72dfb05130ed5f657a32c7250962b2dd1c40fc1132d5f198"}
Aug 02 10:33:55.214867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:55.214702    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2fed233-9b0d-44ad-9778-3ec67dd03656" containerID="68d0ef39be3268f700506a66f885ac431aea83d3eaccb4e939632e6ff37bdbb3" exitCode=0
Aug 02 10:33:55.214867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:55.214753    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rj9b" event={"ID":"a2fed233-9b0d-44ad-9778-3ec67dd03656","Type":"ContainerDied","Data":"68d0ef39be3268f700506a66f885ac431aea83d3eaccb4e939632e6ff37bdbb3"}
Aug 02 10:33:56.219960 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:56.219926    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2fed233-9b0d-44ad-9778-3ec67dd03656" containerID="85873514153efbe7edd1f6d000bb1aa7c040e6e05b9a112d387c466ce61765c2" exitCode=0
Aug 02 10:33:56.220348 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:56.219976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rj9b" event={"ID":"a2fed233-9b0d-44ad-9778-3ec67dd03656","Type":"ContainerDied","Data":"85873514153efbe7edd1f6d000bb1aa7c040e6e05b9a112d387c466ce61765c2"}
Aug 02 10:33:56.592980 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:56.592869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:33:56.593170 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:56.593045    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 02 10:33:56.593170 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:33:56.593117    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls podName:0d7a5405-d4e4-40b8-90c9-06f6b9160b12 nodeName:}" failed. No retries permitted until 2026-08-02 10:34:00.593100563 +0000 UTC m=+42.123683186 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls") pod "dns-default-6hstf" (UID: "0d7a5405-d4e4-40b8-90c9-06f6b9160b12") : secret "dns-default-metrics-tls" not found
Aug 02 10:33:57.227153 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:57.227083    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rj9b" event={"ID":"a2fed233-9b0d-44ad-9778-3ec67dd03656","Type":"ContainerStarted","Data":"cc6d5d41e198c980ac38ff190bfd7f65af050c0f04522d078f8a730900c949f1"}
Aug 02 10:33:57.228803 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:57.228779    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2t452" event={"ID":"9cab4866-7565-40c3-bbc3-c3c7d2e9656a","Type":"ContainerStarted","Data":"ac60ae083e1321b70c5355dd11538e0b9cc80dd914114e413b46e9f8dd5a9af3"}
Aug 02 10:33:57.249430 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:57.249375    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2rj9b" podStartSLOduration=8.64420256 podStartE2EDuration="39.249360332s" totalImagesPullingTime="30.605157772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:57.247802079 +0000 UTC m=+38.778384705" watchObservedRunningTime="2026-08-02 10:33:57.249360332 +0000 UTC m=+38.779942956"
Aug 02 10:33:57.264814 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:33:57.264756    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2t452" podStartSLOduration=2.955762732 podStartE2EDuration="5.264734622s" totalImagesPullingTime="2.30897189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:33:57.264093239 +0000 UTC m=+38.794675895" watchObservedRunningTime="2026-08-02 10:33:57.264734622 +0000 UTC m=+38.795317251"
Aug 02 10:34:00.620564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:00.620525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:34:00.620889 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:00.620675    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 02 10:34:00.620889 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:00.620740    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls podName:0d7a5405-d4e4-40b8-90c9-06f6b9160b12 nodeName:}" failed. No retries permitted until 2026-08-02 10:34:08.620723239 +0000 UTC m=+50.151305845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls") pod "dns-default-6hstf" (UID: "0d7a5405-d4e4-40b8-90c9-06f6b9160b12") : secret "dns-default-metrics-tls" not found
Aug 02 10:34:08.674282 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:08.674240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:34:08.674735 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:08.674391    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 02 10:34:08.674735 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:08.674453    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls podName:0d7a5405-d4e4-40b8-90c9-06f6b9160b12 nodeName:}" failed. No retries permitted until 2026-08-02 10:34:24.674438755 +0000 UTC m=+66.205021363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls") pod "dns-default-6hstf" (UID: "0d7a5405-d4e4-40b8-90c9-06f6b9160b12") : secret "dns-default-metrics-tls" not found
Aug 02 10:34:19.210460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:19.210427    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j98mk"
Aug 02 10:34:23.572238 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.572184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:34:23.575187 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.575166    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 02 10:34:23.584799 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.584783    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 02 10:34:23.596309 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.596284    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfgw5\" (UniqueName: \"kubernetes.io/projected/b2ec253c-efc7-49ee-a2da-5fcd3453c36a-kube-api-access-pfgw5\") pod \"network-check-target-r8c9h\" (UID: \"b2ec253c-efc7-49ee-a2da-5fcd3453c36a\") " pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:34:23.665884 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.665855    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q9bgf\""
Aug 02 10:34:23.673344 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.673308    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:34:23.673419 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.673369    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:34:23.673460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.673412    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:34:23.676167 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.676150    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 02 10:34:23.676207 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.676151    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 02 10:34:23.683914 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:23.683882    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 02 10:34:23.684089 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:23.683955    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:35:27.683934548 +0000 UTC m=+129.214517157 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : secret "metrics-daemon-secret" not found
Aug 02 10:34:23.686377 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.686355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e17cf15b-0d83-4f4d-afa8-23a6c52838f8-original-pull-secret\") pod \"global-pull-secret-syncer-xfdxj\" (UID: \"e17cf15b-0d83-4f4d-afa8-23a6c52838f8\") " pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:34:23.797666 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.797637    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r8c9h"]
Aug 02 10:34:23.800576 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:34:23.800549    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ec253c_efc7_49ee_a2da_5fcd3453c36a.slice/crio-85f177af47932a2770da6363249a62f39b43c905ecd7130c41d3d7318f51a837 WatchSource:0}: Error finding container 85f177af47932a2770da6363249a62f39b43c905ecd7130c41d3d7318f51a837: Status 404 returned error can't find the container with id 85f177af47932a2770da6363249a62f39b43c905ecd7130c41d3d7318f51a837
Aug 02 10:34:23.968475 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:23.968447    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfdxj"
Aug 02 10:34:24.086776 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:24.086744    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xfdxj"]
Aug 02 10:34:24.089864 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:34:24.089841    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode17cf15b_0d83_4f4d_afa8_23a6c52838f8.slice/crio-1cf956ea5a75f9c325307d353112a6bd4da74b91ef9f0e199d03338461a34861 WatchSource:0}: Error finding container 1cf956ea5a75f9c325307d353112a6bd4da74b91ef9f0e199d03338461a34861: Status 404 returned error can't find the container with id 1cf956ea5a75f9c325307d353112a6bd4da74b91ef9f0e199d03338461a34861
Aug 02 10:34:24.289314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:24.289230    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xfdxj" event={"ID":"e17cf15b-0d83-4f4d-afa8-23a6c52838f8","Type":"ContainerStarted","Data":"1cf956ea5a75f9c325307d353112a6bd4da74b91ef9f0e199d03338461a34861"}
Aug 02 10:34:24.290505 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:24.290473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r8c9h" event={"ID":"b2ec253c-efc7-49ee-a2da-5fcd3453c36a","Type":"ContainerStarted","Data":"85f177af47932a2770da6363249a62f39b43c905ecd7130c41d3d7318f51a837"}
Aug 02 10:34:24.681648 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:24.681600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:34:24.682088 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:24.681719    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 02 10:34:24.682088 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:24.681788    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls podName:0d7a5405-d4e4-40b8-90c9-06f6b9160b12 nodeName:}" failed. No retries permitted until 2026-08-02 10:34:56.681767343 +0000 UTC m=+98.212349949 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls") pod "dns-default-6hstf" (UID: "0d7a5405-d4e4-40b8-90c9-06f6b9160b12") : secret "dns-default-metrics-tls" not found
Aug 02 10:34:28.302567 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:28.302408    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xfdxj" event={"ID":"e17cf15b-0d83-4f4d-afa8-23a6c52838f8","Type":"ContainerStarted","Data":"612c6757bec51bf733d5badd7d838a2277166bb2a9404f294edbf34c3d19e475"}
Aug 02 10:34:28.304123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:28.304096    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r8c9h" event={"ID":"b2ec253c-efc7-49ee-a2da-5fcd3453c36a","Type":"ContainerStarted","Data":"7954fe0ba966c14166f302c6ef0c15f729c2b773324ac065290ce97fa0489b01"}
Aug 02 10:34:28.304242 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:28.304216    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:34:28.318136 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:28.318081    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xfdxj" podStartSLOduration=66.206804684 podStartE2EDuration="1m10.318062101s" totalImagesPullingTime="4.111257417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:34:28.316855951 +0000 UTC m=+69.847438590" watchObservedRunningTime="2026-08-02 10:34:28.318062101 +0000 UTC m=+69.848644728"
Aug 02 10:34:28.331935 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:28.331886    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r8c9h" podStartSLOduration=65.93505462 podStartE2EDuration="1m10.331872749s" totalImagesPullingTime="4.396818129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:34:28.331310844 +0000 UTC m=+69.861893474" watchObservedRunningTime="2026-08-02 10:34:28.331872749 +0000 UTC m=+69.862455378"
Aug 02 10:34:56.695946 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:56.695832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:34:56.696418 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:56.695951    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 02 10:34:56.696418 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:34:56.696003    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls podName:0d7a5405-d4e4-40b8-90c9-06f6b9160b12 nodeName:}" failed. No retries permitted until 2026-08-02 10:36:00.695990381 +0000 UTC m=+162.226572989 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls") pod "dns-default-6hstf" (UID: "0d7a5405-d4e4-40b8-90c9-06f6b9160b12") : secret "dns-default-metrics-tls" not found
Aug 02 10:34:59.308919 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:34:59.308891    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r8c9h"
Aug 02 10:35:27.697504 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:35:27.697463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:35:27.698081 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:35:27.697606    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 02 10:35:27.698081 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:35:27.697697    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:37:29.697674606 +0000 UTC m=+251.228257214 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : secret "metrics-daemon-secret" not found
Aug 02 10:35:55.838849 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:35:55.838796    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6hstf" podUID="0d7a5405-d4e4-40b8-90c9-06f6b9160b12"
Aug 02 10:35:56.505420 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:35:56.505389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6hstf"
Aug 02 10:35:56.956449 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:35:56.956417    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:36:00.715618 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:36:00.715581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:36:00.716005 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:36:00.715729    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 02 10:36:00.716005 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:36:00.715796    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls podName:0d7a5405-d4e4-40b8-90c9-06f6b9160b12 nodeName:}" failed. No retries permitted until 2026-08-02 10:38:02.715781072 +0000 UTC m=+284.246363682 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls") pod "dns-default-6hstf" (UID: "0d7a5405-d4e4-40b8-90c9-06f6b9160b12") : secret "dns-default-metrics-tls" not found
Aug 02 10:36:08.947562 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:36:08.947535    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:37:29.707637 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:29.707592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:37:29.708203 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:29.707727    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 02 10:37:29.708203 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:29.707816    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs podName:7f208a37-3b97-4072-ac20-9ffbf8732d84 nodeName:}" failed. No retries permitted until 2026-08-02 10:39:31.707795515 +0000 UTC m=+373.238378141 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs") pod "network-metrics-daemon-mmf2n" (UID: "7f208a37-3b97-4072-ac20-9ffbf8732d84") : secret "metrics-daemon-secret" not found
Aug 02 10:37:33.267150 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.267116    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq"]
Aug 02 10:37:33.269898 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.269880    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq"
Aug 02 10:37:33.272584 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.272565    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Aug 02 10:37:33.273579 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.273556    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Aug 02 10:37:33.273661 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.273565    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-dt6jc\""
Aug 02 10:37:33.289942 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.289916    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq"]
Aug 02 10:37:33.330471 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.330403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbjs\" (UniqueName: \"kubernetes.io/projected/f63fbd96-806b-4a29-8c20-77f857e4c36d-kube-api-access-lpbjs\") pod \"volume-data-source-validator-5b8776669-l5dsq\" (UID: \"f63fbd96-806b-4a29-8c20-77f857e4c36d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq"
Aug 02 10:37:33.373879 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.373852    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6df46b759b-kktpn"]
Aug 02 10:37:33.376469 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.376455    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-b498cf56b-948pc"]
Aug 02 10:37:33.376586 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.376568    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.379423 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.379401    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.379510 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.379440    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 02 10:37:33.379598 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.379578    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 02 10:37:33.380101 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.380079    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-plb6r\""
Aug 02 10:37:33.380179 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.380156    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 02 10:37:33.383098 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.383078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Aug 02 10:37:33.383194 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.383097    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Aug 02 10:37:33.383775 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.383742    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Aug 02 10:37:33.385769 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.385750    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Aug 02 10:37:33.385869 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.385786    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cxkr9\""
Aug 02 10:37:33.385869 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.385806    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 02 10:37:33.385869 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.385827    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 02 10:37:33.385993 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.385903    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 02 10:37:33.396639 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.396621    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b498cf56b-948pc"]
Aug 02 10:37:33.397379 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.397364    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6df46b759b-kktpn"]
Aug 02 10:37:33.430904 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.430876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.431049 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.430914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zl47\" (UniqueName: \"kubernetes.io/projected/f663257d-867f-4ff1-8660-9ea9584d555e-kube-api-access-9zl47\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.431049 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.430946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbjs\" (UniqueName: \"kubernetes.io/projected/f63fbd96-806b-4a29-8c20-77f857e4c36d-kube-api-access-lpbjs\") pod \"volume-data-source-validator-5b8776669-l5dsq\" (UID: \"f63fbd96-806b-4a29-8c20-77f857e4c36d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq"
Aug 02 10:37:33.431049 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.430970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-stats-auth\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.431049 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-image-registry-private-configuration\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.431237 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431051    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-trusted-ca\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.431237 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431077    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c27b5d62-9959-434c-b16f-edf16b7498e8-ca-trust-extracted\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.431237 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-certificates\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.431237 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-bound-sa-token\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.431237 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431157    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvkzr\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-kube-api-access-hvkzr\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.431237 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.431421 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431245    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.431421 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-default-certificate\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.431421 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.431311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-installation-pull-secrets\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.438598 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.438579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbjs\" (UniqueName: \"kubernetes.io/projected/f63fbd96-806b-4a29-8c20-77f857e4c36d-kube-api-access-lpbjs\") pod \"volume-data-source-validator-5b8776669-l5dsq\" (UID: \"f63fbd96-806b-4a29-8c20-77f857e4c36d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq"
Aug 02 10:37:33.531737 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-default-certificate\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.531737 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531726    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-installation-pull-secrets\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.531890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.531890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531768    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zl47\" (UniqueName: \"kubernetes.io/projected/f663257d-867f-4ff1-8660-9ea9584d555e-kube-api-access-9zl47\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.531890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-stats-auth\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.531890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-image-registry-private-configuration\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.531890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531829    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-trusted-ca\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.531890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c27b5d62-9959-434c-b16f-edf16b7498e8-ca-trust-extracted\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.531890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-certificates\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.532259 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531901    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-bound-sa-token\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.532259 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvkzr\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-kube-api-access-hvkzr\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.532259 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:33.531940    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 02 10:37:33.532259 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.532259 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.531988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.532259 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:33.532022    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:34.03200182 +0000 UTC m=+255.562584441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : secret "router-metrics-certs-default" not found
Aug 02 10:37:33.532259 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:33.532108    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 02 10:37:33.532259 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:33.532123    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6df46b759b-kktpn: secret "image-registry-tls" not found
Aug 02 10:37:33.532259 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:33.532179    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls podName:c27b5d62-9959-434c-b16f-edf16b7498e8 nodeName:}" failed. No retries permitted until 2026-08-02 10:37:34.032163759 +0000 UTC m=+255.562746384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls") pod "image-registry-6df46b759b-kktpn" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8") : secret "image-registry-tls" not found
Aug 02 10:37:33.532687 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.532267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c27b5d62-9959-434c-b16f-edf16b7498e8-ca-trust-extracted\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.532687 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:33.532591    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:34.03257531 +0000 UTC m=+255.563157919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : configmap references non-existent config key: service-ca.crt
Aug 02 10:37:33.532992 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.532969    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-certificates\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.533512 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.533489    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-trusted-ca\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.534554 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.534529    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-default-certificate\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.534698 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.534677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-stats-auth\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.534774 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.534713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-image-registry-private-configuration\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.534848 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.534827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-installation-pull-secrets\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.541276 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.541253    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zl47\" (UniqueName: \"kubernetes.io/projected/f663257d-867f-4ff1-8660-9ea9584d555e-kube-api-access-9zl47\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:33.541919 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.541899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-bound-sa-token\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.542235 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.542219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvkzr\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-kube-api-access-hvkzr\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:33.577888 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.577852    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq"
Aug 02 10:37:33.701407 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.701380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq"]
Aug 02 10:37:33.704380 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:37:33.704357    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf63fbd96_806b_4a29_8c20_77f857e4c36d.slice/crio-3f71f232499275816187fc95d4c3906cc8ca90499d143886357697dd09cac279 WatchSource:0}: Error finding container 3f71f232499275816187fc95d4c3906cc8ca90499d143886357697dd09cac279: Status 404 returned error can't find the container with id 3f71f232499275816187fc95d4c3906cc8ca90499d143886357697dd09cac279
Aug 02 10:37:33.736446 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:33.736419    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq" event={"ID":"f63fbd96-806b-4a29-8c20-77f857e4c36d","Type":"ContainerStarted","Data":"3f71f232499275816187fc95d4c3906cc8ca90499d143886357697dd09cac279"}
Aug 02 10:37:34.035762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:34.035729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:34.035930 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:34.035782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:34.035930 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:34.035805    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:34.035930 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:34.035880    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 02 10:37:34.035930 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:34.035898    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 02 10:37:34.035930 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:34.035910    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6df46b759b-kktpn: secret "image-registry-tls" not found
Aug 02 10:37:34.035930 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:34.035921    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:35.035901893 +0000 UTC m=+256.566484500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : configmap references non-existent config key: service-ca.crt
Aug 02 10:37:34.036148 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:34.035944    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls podName:c27b5d62-9959-434c-b16f-edf16b7498e8 nodeName:}" failed. No retries permitted until 2026-08-02 10:37:35.035933511 +0000 UTC m=+256.566516134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls") pod "image-registry-6df46b759b-kktpn" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8") : secret "image-registry-tls" not found
Aug 02 10:37:34.036148 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:34.035957    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:35.035950225 +0000 UTC m=+256.566532834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : secret "router-metrics-certs-default" not found
Aug 02 10:37:35.043755 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:35.043717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:35.044205 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:35.043778    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:35.044205 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:35.043796    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:35.044205 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:35.043861    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 02 10:37:35.044205 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:35.043882    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 02 10:37:35.044205 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:35.043892    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6df46b759b-kktpn: secret "image-registry-tls" not found
Aug 02 10:37:35.044205 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:35.043928    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:37.043911549 +0000 UTC m=+258.574494156 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : secret "router-metrics-certs-default" not found
Aug 02 10:37:35.044205 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:35.043944    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls podName:c27b5d62-9959-434c-b16f-edf16b7498e8 nodeName:}" failed. No retries permitted until 2026-08-02 10:37:37.043936266 +0000 UTC m=+258.574518872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls") pod "image-registry-6df46b759b-kktpn" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8") : secret "image-registry-tls" not found
Aug 02 10:37:35.044205 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:35.043954    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:37.043948557 +0000 UTC m=+258.574531165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : configmap references non-existent config key: service-ca.crt
Aug 02 10:37:35.743179 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:35.743138    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq" event={"ID":"f63fbd96-806b-4a29-8c20-77f857e4c36d","Type":"ContainerStarted","Data":"8c0c0f27246a626fb97cf0c5877494cc17d4447cd986aa53ba8232e912d7d052"}
Aug 02 10:37:35.774917 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:35.774874    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-l5dsq" podStartSLOduration=1.649969937 podStartE2EDuration="2.774859455s" totalImagesPullingTime="1.124889518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:37:35.774587863 +0000 UTC m=+257.305170504" watchObservedRunningTime="2026-08-02 10:37:35.774859455 +0000 UTC m=+257.305442085"
Aug 02 10:37:37.059674 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.059611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:37.059674 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.059668    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:37.060141 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.059754    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 02 10:37:37.060141 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.059765    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6df46b759b-kktpn: secret "image-registry-tls" not found
Aug 02 10:37:37.060141 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.059779    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:41.059761392 +0000 UTC m=+262.590344003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : configmap references non-existent config key: service-ca.crt
Aug 02 10:37:37.060141 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.059804    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls podName:c27b5d62-9959-434c-b16f-edf16b7498e8 nodeName:}" failed. No retries permitted until 2026-08-02 10:37:41.05979283 +0000 UTC m=+262.590375436 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls") pod "image-registry-6df46b759b-kktpn" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8") : secret "image-registry-tls" not found
Aug 02 10:37:37.060141 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.059835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:37.060141 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.059888    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 02 10:37:37.060141 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.059909    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:41.059903018 +0000 UTC m=+262.590485625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : secret "router-metrics-certs-default" not found
Aug 02 10:37:37.384936 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.384862    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gqnd5"]
Aug 02 10:37:37.387996 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.387978    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.390535 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.390517    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 02 10:37:37.390632 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.390516    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 02 10:37:37.390812 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.390797    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 02 10:37:37.391755 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.391736    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 02 10:37:37.391831 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.391778    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-94g6g\""
Aug 02 10:37:37.398067 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.398047    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gqnd5"]
Aug 02 10:37:37.462405 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.462385    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a662df23-b5df-4447-b0bf-713a62e1770a-crio-socket\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.462518 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.462411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a662df23-b5df-4447-b0bf-713a62e1770a-data-volume\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.462518 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.462450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.462518 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.462508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.462615 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.462533    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw8rm\" (UniqueName: \"kubernetes.io/projected/a662df23-b5df-4447-b0bf-713a62e1770a-kube-api-access-sw8rm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.563680 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.563642    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a662df23-b5df-4447-b0bf-713a62e1770a-crio-socket\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.563680 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.563682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a662df23-b5df-4447-b0bf-713a62e1770a-data-volume\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.563789 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.563761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a662df23-b5df-4447-b0bf-713a62e1770a-crio-socket\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.563822 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.563797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.563854 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.563842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.563889 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.563858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw8rm\" (UniqueName: \"kubernetes.io/projected/a662df23-b5df-4447-b0bf-713a62e1770a-kube-api-access-sw8rm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.563963 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.563950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a662df23-b5df-4447-b0bf-713a62e1770a-data-volume\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:37.563997 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.563964    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:37.563997 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.563979    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 02 10:37:37.564091 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.564041    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:38.064011489 +0000 UTC m=+259.594594095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:37.564091 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:37.564065    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:38.064048824 +0000 UTC m=+259.594631445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : configmap "kube-rbac-proxy" not found
Aug 02 10:37:37.572847 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:37.572828    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw8rm\" (UniqueName: \"kubernetes.io/projected/a662df23-b5df-4447-b0bf-713a62e1770a-kube-api-access-sw8rm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:38.067719 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:38.067671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:38.068133 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:38.067735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:38.068133 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:38.067806    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 02 10:37:38.068133 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:38.067819    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:38.068133 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:38.067877    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:39.067863418 +0000 UTC m=+260.598446024 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : configmap "kube-rbac-proxy" not found
Aug 02 10:37:38.068133 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:38.067889    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:39.067883297 +0000 UTC m=+260.598465905 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:39.076305 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:39.076273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:39.076685 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:39.076355    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:39.076685 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:39.076396    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 02 10:37:39.076685 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:39.076447    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:39.076685 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:39.076455    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:41.076440888 +0000 UTC m=+262.607023496 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : configmap "kube-rbac-proxy" not found
Aug 02 10:37:39.076685 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:39.076484    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:41.076473247 +0000 UTC m=+262.607055854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:40.345633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.345561    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkcz8_a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786/dns-node-resolver/0.log"
Aug 02 10:37:40.388634 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.388611    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-6qbkk"]
Aug 02 10:37:40.391319 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.391305    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.393985 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.393969    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Aug 02 10:37:40.394169 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.394153    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Aug 02 10:37:40.395241 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.395226    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Aug 02 10:37:40.395535 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.395517    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Aug 02 10:37:40.395607 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.395518    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-448mj\""
Aug 02 10:37:40.398790 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.398770    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-6qbkk"]
Aug 02 10:37:40.486700 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.486669    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz8rv\" (UniqueName: \"kubernetes.io/projected/ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7-kube-api-access-mz8rv\") pod \"service-ca-98c88fbb5-6qbkk\" (UID: \"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7\") " pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.486838 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.486760    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7-signing-key\") pod \"service-ca-98c88fbb5-6qbkk\" (UID: \"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7\") " pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.486838 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.486810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7-signing-cabundle\") pod \"service-ca-98c88fbb5-6qbkk\" (UID: \"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7\") " pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.587248 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.587221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7-signing-cabundle\") pod \"service-ca-98c88fbb5-6qbkk\" (UID: \"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7\") " pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.587382 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.587277    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz8rv\" (UniqueName: \"kubernetes.io/projected/ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7-kube-api-access-mz8rv\") pod \"service-ca-98c88fbb5-6qbkk\" (UID: \"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7\") " pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.587382 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.587326    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7-signing-key\") pod \"service-ca-98c88fbb5-6qbkk\" (UID: \"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7\") " pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.587974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.587907    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7-signing-cabundle\") pod \"service-ca-98c88fbb5-6qbkk\" (UID: \"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7\") " pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.589709 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.589688    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7-signing-key\") pod \"service-ca-98c88fbb5-6qbkk\" (UID: \"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7\") " pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.596287 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.596244    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz8rv\" (UniqueName: \"kubernetes.io/projected/ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7-kube-api-access-mz8rv\") pod \"service-ca-98c88fbb5-6qbkk\" (UID: \"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7\") " pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.701309 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.701283    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk"
Aug 02 10:37:40.819710 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:40.819681    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-6qbkk"]
Aug 02 10:37:40.822692 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:37:40.822667    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf3c834_56d9_49b5_91d5_f3a0ca46e0d7.slice/crio-4fdf02692d548edff1706ed6f8cf20fbd3ab06df1a023e2c41873d6555ccb3c5 WatchSource:0}: Error finding container 4fdf02692d548edff1706ed6f8cf20fbd3ab06df1a023e2c41873d6555ccb3c5: Status 404 returned error can't find the container with id 4fdf02692d548edff1706ed6f8cf20fbd3ab06df1a023e2c41873d6555ccb3c5
Aug 02 10:37:41.091346 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:41.091308    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:41.091535 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:41.091364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:41.091535 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:41.091394    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:41.091535 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:41.091420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:41.091535 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091455    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 02 10:37:41.091535 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091493    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:41.091535 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091530    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:45.091510327 +0000 UTC m=+266.622092940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : configmap "kube-rbac-proxy" not found
Aug 02 10:37:41.091767 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091538    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 02 10:37:41.091767 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091535    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 02 10:37:41.091767 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:41.091459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:41.091767 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091574    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6df46b759b-kktpn: secret "image-registry-tls" not found
Aug 02 10:37:41.091767 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091551    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:45.091541603 +0000 UTC m=+266.622124212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:41.091767 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091637    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:49.091619016 +0000 UTC m=+270.622201625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : secret "router-metrics-certs-default" not found
Aug 02 10:37:41.091767 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091653    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:37:49.091646255 +0000 UTC m=+270.622228862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : configmap references non-existent config key: service-ca.crt
Aug 02 10:37:41.091767 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:41.091664    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls podName:c27b5d62-9959-434c-b16f-edf16b7498e8 nodeName:}" failed. No retries permitted until 2026-08-02 10:37:49.091659148 +0000 UTC m=+270.622241754 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls") pod "image-registry-6df46b759b-kktpn" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8") : secret "image-registry-tls" not found
Aug 02 10:37:41.141644 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:41.141620    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fbkmq_92f58dc5-7223-40a2-85bb-020e1ef6d6a1/node-ca/0.log"
Aug 02 10:37:41.743062 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:41.743013    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2t452_9cab4866-7565-40c3-bbc3-c3c7d2e9656a/serve-healthcheck-canary/0.log"
Aug 02 10:37:41.759694 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:41.759660    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk" event={"ID":"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7","Type":"ContainerStarted","Data":"4fdf02692d548edff1706ed6f8cf20fbd3ab06df1a023e2c41873d6555ccb3c5"}
Aug 02 10:37:42.763889 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:42.763808    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk" event={"ID":"ecf3c834-56d9-49b5-91d5-f3a0ca46e0d7","Type":"ContainerStarted","Data":"37d7aa9c1c56045b07d7c9d3ed63347e391c153c068de1fd72562744d60cc7f1"}
Aug 02 10:37:42.780613 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:42.780562    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-6qbkk" podStartSLOduration=1.152245828 podStartE2EDuration="2.780546745s" totalImagesPullingTime="1.628300917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:37:42.779775533 +0000 UTC m=+264.310358162" watchObservedRunningTime="2026-08-02 10:37:42.780546745 +0000 UTC m=+264.311129394"
Aug 02 10:37:45.124896 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:45.124866    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:45.125402 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:45.124918    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:45.125402 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:45.124996    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 02 10:37:45.125402 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:45.125065    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:45.125402 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:45.125072    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:53.125054292 +0000 UTC m=+274.655636913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : configmap "kube-rbac-proxy" not found
Aug 02 10:37:45.125402 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:45.125101    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:37:53.125091371 +0000 UTC m=+274.655673979 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:49.156103 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.156061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:49.156538 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.156155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:37:49.156538 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.156172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:49.156538 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:49.156228    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 02 10:37:49.156538 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:49.156301    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:38:05.15628462 +0000 UTC m=+286.686867230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : configmap references non-existent config key: service-ca.crt
Aug 02 10:37:49.156538 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:49.156321    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs podName:f663257d-867f-4ff1-8660-9ea9584d555e nodeName:}" failed. No retries permitted until 2026-08-02 10:38:05.156314177 +0000 UTC m=+286.686896785 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs") pod "router-default-b498cf56b-948pc" (UID: "f663257d-867f-4ff1-8660-9ea9584d555e") : secret "router-metrics-certs-default" not found
Aug 02 10:37:49.158700 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.158678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") pod \"image-registry-6df46b759b-kktpn\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") " pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:49.287135 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.287104    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:49.454946 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.454855    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6df46b759b-kktpn"]
Aug 02 10:37:49.459904 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:37:49.459866    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27b5d62_9959_434c_b16f_edf16b7498e8.slice/crio-0e156856048806844e9af7940e89e33652d295e704456d1f49973e975903cbe8 WatchSource:0}: Error finding container 0e156856048806844e9af7940e89e33652d295e704456d1f49973e975903cbe8: Status 404 returned error can't find the container with id 0e156856048806844e9af7940e89e33652d295e704456d1f49973e975903cbe8
Aug 02 10:37:49.791325 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.791240    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6df46b759b-kktpn" event={"ID":"c27b5d62-9959-434c-b16f-edf16b7498e8","Type":"ContainerStarted","Data":"bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c"}
Aug 02 10:37:49.791325 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.791295    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6df46b759b-kktpn" event={"ID":"c27b5d62-9959-434c-b16f-edf16b7498e8","Type":"ContainerStarted","Data":"0e156856048806844e9af7940e89e33652d295e704456d1f49973e975903cbe8"}
Aug 02 10:37:49.791547 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.791369    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:37:49.812146 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:49.812092    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6df46b759b-kktpn" podStartSLOduration=16.812077316 podStartE2EDuration="16.812077316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:37:49.810463501 +0000 UTC m=+271.341046149" watchObservedRunningTime="2026-08-02 10:37:49.812077316 +0000 UTC m=+271.342659945"
Aug 02 10:37:53.185564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:53.185534    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:53.186138 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:53.185617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:53.186138 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:53.185707    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:53.186138 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:53.185787    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls podName:a662df23-b5df-4447-b0bf-713a62e1770a nodeName:}" failed. No retries permitted until 2026-08-02 10:38:09.185766763 +0000 UTC m=+290.716349370 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqnd5" (UID: "a662df23-b5df-4447-b0bf-713a62e1770a") : secret "insights-runtime-extractor-tls" not found
Aug 02 10:37:53.186138 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:53.186110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a662df23-b5df-4447-b0bf-713a62e1770a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:37:59.506863 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:37:59.506814    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6hstf" podUID="0d7a5405-d4e4-40b8-90c9-06f6b9160b12"
Aug 02 10:37:59.820819 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:37:59.820745    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6hstf"
Aug 02 10:38:02.754360 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:02.754324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:38:02.756680 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:02.756655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d7a5405-d4e4-40b8-90c9-06f6b9160b12-metrics-tls\") pod \"dns-default-6hstf\" (UID: \"0d7a5405-d4e4-40b8-90c9-06f6b9160b12\") " pod="openshift-dns/dns-default-6hstf"
Aug 02 10:38:02.824642 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:02.824616    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nv7kc\""
Aug 02 10:38:02.833008 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:02.832979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6hstf"
Aug 02 10:38:02.847605 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:02.847582    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6df46b759b-kktpn"]
Aug 02 10:38:02.979325 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:02.979295    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6hstf"]
Aug 02 10:38:02.982381 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:38:02.982354    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d7a5405_d4e4_40b8_90c9_06f6b9160b12.slice/crio-1dc76bd225b01c67b7d89fdc212ed318498078d3ca93a0db6dc6c1e2ab37a379 WatchSource:0}: Error finding container 1dc76bd225b01c67b7d89fdc212ed318498078d3ca93a0db6dc6c1e2ab37a379: Status 404 returned error can't find the container with id 1dc76bd225b01c67b7d89fdc212ed318498078d3ca93a0db6dc6c1e2ab37a379
Aug 02 10:38:03.832762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:03.832719    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6hstf" event={"ID":"0d7a5405-d4e4-40b8-90c9-06f6b9160b12","Type":"ContainerStarted","Data":"1dc76bd225b01c67b7d89fdc212ed318498078d3ca93a0db6dc6c1e2ab37a379"}
Aug 02 10:38:04.837293 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:04.837206    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6hstf" event={"ID":"0d7a5405-d4e4-40b8-90c9-06f6b9160b12","Type":"ContainerStarted","Data":"4d594cbb92ebf2cadeb26a70d3cf562ebf619371d51f69e1da5b0e4ada853a0f"}
Aug 02 10:38:04.837293 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:04.837289    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6hstf" event={"ID":"0d7a5405-d4e4-40b8-90c9-06f6b9160b12","Type":"ContainerStarted","Data":"8fe960cb2b9e19e477134d59b47fb0bc54420629cab489e924ca65843f24d776"}
Aug 02 10:38:04.837703 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:04.837326    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6hstf"
Aug 02 10:38:04.855375 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:04.855329    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6hstf" podStartSLOduration=251.258036989 podStartE2EDuration="4m12.855315218s" totalImagesPullingTime="1.597278229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:38:04.853649806 +0000 UTC m=+286.384232454" watchObservedRunningTime="2026-08-02 10:38:04.855315218 +0000 UTC m=+286.385897847"
Aug 02 10:38:05.174228 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:05.174195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:38:05.174382 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:05.174302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:38:05.174927 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:05.174909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f663257d-867f-4ff1-8660-9ea9584d555e-service-ca-bundle\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:38:05.176784 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:05.176767    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f663257d-867f-4ff1-8660-9ea9584d555e-metrics-certs\") pod \"router-default-b498cf56b-948pc\" (UID: \"f663257d-867f-4ff1-8660-9ea9584d555e\") " pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:38:05.191734 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:05.191691    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:38:05.314486 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:05.314439    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b498cf56b-948pc"]
Aug 02 10:38:05.317816 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:38:05.317791    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf663257d_867f_4ff1_8660_9ea9584d555e.slice/crio-c666c6036d0d66e3bdf8fa63e67dc0fa7e93575a07d24cd727f961eb15069c52 WatchSource:0}: Error finding container c666c6036d0d66e3bdf8fa63e67dc0fa7e93575a07d24cd727f961eb15069c52: Status 404 returned error can't find the container with id c666c6036d0d66e3bdf8fa63e67dc0fa7e93575a07d24cd727f961eb15069c52
Aug 02 10:38:05.842118 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:05.842079    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b498cf56b-948pc" event={"ID":"f663257d-867f-4ff1-8660-9ea9584d555e","Type":"ContainerStarted","Data":"6f30974a82366b7aec7d95531fdadfbf3b02373ac2368248d39ec21b78ede9ec"}
Aug 02 10:38:05.842566 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:05.842132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b498cf56b-948pc" event={"ID":"f663257d-867f-4ff1-8660-9ea9584d555e","Type":"ContainerStarted","Data":"c666c6036d0d66e3bdf8fa63e67dc0fa7e93575a07d24cd727f961eb15069c52"}
Aug 02 10:38:05.861384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:05.861344    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-b498cf56b-948pc" podStartSLOduration=32.861328997 podStartE2EDuration="32.861328997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:38:05.860733082 +0000 UTC m=+287.391315723" watchObservedRunningTime="2026-08-02 10:38:05.861328997 +0000 UTC m=+287.391911625"
Aug 02 10:38:06.191835 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:06.191806    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:38:06.194433 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:06.194410    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:38:06.845407 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:06.845376    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:38:06.846671 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:06.846650    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-b498cf56b-948pc"
Aug 02 10:38:09.204514 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:09.204470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:38:09.206999 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:09.206972    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a662df23-b5df-4447-b0bf-713a62e1770a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqnd5\" (UID: \"a662df23-b5df-4447-b0bf-713a62e1770a\") " pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:38:09.496671 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:09.496575    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gqnd5"
Aug 02 10:38:09.619433 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:09.619398    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gqnd5"]
Aug 02 10:38:09.622198 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:38:09.622174    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda662df23_b5df_4447_b0bf_713a62e1770a.slice/crio-1d77b9551b6db3660724dc2a1dca2051a6698e1646a7f298f74f026833ac2188 WatchSource:0}: Error finding container 1d77b9551b6db3660724dc2a1dca2051a6698e1646a7f298f74f026833ac2188: Status 404 returned error can't find the container with id 1d77b9551b6db3660724dc2a1dca2051a6698e1646a7f298f74f026833ac2188
Aug 02 10:38:09.855327 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:09.855239    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gqnd5" event={"ID":"a662df23-b5df-4447-b0bf-713a62e1770a","Type":"ContainerStarted","Data":"f879b0b9496947f35c249c75f327d67c943acafb0d794f1024926a213dd45e77"}
Aug 02 10:38:09.855327 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:09.855295    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gqnd5" event={"ID":"a662df23-b5df-4447-b0bf-713a62e1770a","Type":"ContainerStarted","Data":"1d77b9551b6db3660724dc2a1dca2051a6698e1646a7f298f74f026833ac2188"}
Aug 02 10:38:10.860207 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:10.860172    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gqnd5" event={"ID":"a662df23-b5df-4447-b0bf-713a62e1770a","Type":"ContainerStarted","Data":"6f3b810e09190de25e1114d8d8b6f70a816fec4e8ccce3c116fdf92d7c93a8a7"}
Aug 02 10:38:11.102261 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.102218    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"]
Aug 02 10:38:11.105509 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.105488    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.109436 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.109408    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Aug 02 10:38:11.109867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.109652    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Aug 02 10:38:11.109867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.109660    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 02 10:38:11.109867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.109659    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 02 10:38:11.109867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.109774    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 02 10:38:11.109867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.109674    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hrl2j\""
Aug 02 10:38:11.114787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.114766    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"]
Aug 02 10:38:11.219958 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.219923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b61d89c3-54aa-4f47-aeb6-e45095114e7b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.219958 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.219961    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b61d89c3-54aa-4f47-aeb6-e45095114e7b-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.220196 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.220051    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61d89c3-54aa-4f47-aeb6-e45095114e7b-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.220196 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.220101    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ls68\" (UniqueName: \"kubernetes.io/projected/b61d89c3-54aa-4f47-aeb6-e45095114e7b-kube-api-access-8ls68\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.320712 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.320676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b61d89c3-54aa-4f47-aeb6-e45095114e7b-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.320895 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.320721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61d89c3-54aa-4f47-aeb6-e45095114e7b-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.320895 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.320756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ls68\" (UniqueName: \"kubernetes.io/projected/b61d89c3-54aa-4f47-aeb6-e45095114e7b-kube-api-access-8ls68\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.320895 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.320810    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b61d89c3-54aa-4f47-aeb6-e45095114e7b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.321432 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.321409    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b61d89c3-54aa-4f47-aeb6-e45095114e7b-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.323497 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.323474    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61d89c3-54aa-4f47-aeb6-e45095114e7b-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.323599 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.323545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b61d89c3-54aa-4f47-aeb6-e45095114e7b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.329319 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.329303    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ls68\" (UniqueName: \"kubernetes.io/projected/b61d89c3-54aa-4f47-aeb6-e45095114e7b-kube-api-access-8ls68\") pod \"prometheus-operator-758d7cb874-hhd8z\" (UID: \"b61d89c3-54aa-4f47-aeb6-e45095114e7b\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.418266 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.418235    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"
Aug 02 10:38:11.785592 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.785549    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-hhd8z"]
Aug 02 10:38:11.788618 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:38:11.788598    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb61d89c3_54aa_4f47_aeb6_e45095114e7b.slice/crio-2879a5b4d9e844c09d618d4c12c9d27caebf795284611e4705ee1a93a79f822a WatchSource:0}: Error finding container 2879a5b4d9e844c09d618d4c12c9d27caebf795284611e4705ee1a93a79f822a: Status 404 returned error can't find the container with id 2879a5b4d9e844c09d618d4c12c9d27caebf795284611e4705ee1a93a79f822a
Aug 02 10:38:11.865042 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.864982    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gqnd5" event={"ID":"a662df23-b5df-4447-b0bf-713a62e1770a","Type":"ContainerStarted","Data":"4558ccd5b50d9c1da8e6d456e80629bcac8945bcfa71911130f9511311cababb"}
Aug 02 10:38:11.866091 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.866063    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z" event={"ID":"b61d89c3-54aa-4f47-aeb6-e45095114e7b","Type":"ContainerStarted","Data":"2879a5b4d9e844c09d618d4c12c9d27caebf795284611e4705ee1a93a79f822a"}
Aug 02 10:38:11.884684 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:11.884624    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gqnd5" podStartSLOduration=32.908934958 podStartE2EDuration="34.884605125s" totalImagesPullingTime="1.975670167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 10:37:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:38:11.883692436 +0000 UTC m=+293.414275066" watchObservedRunningTime="2026-08-02 10:38:11.884605125 +0000 UTC m=+293.415187755"
Aug 02 10:38:11.948419 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:38:11.948382    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mmf2n" podUID="7f208a37-3b97-4072-ac20-9ffbf8732d84"
Aug 02 10:38:12.855204 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:12.855179    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:38:13.874322 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:13.874286    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z" event={"ID":"b61d89c3-54aa-4f47-aeb6-e45095114e7b","Type":"ContainerStarted","Data":"7fa549bb43641e5956636587c44900964246b52983028fb843f53279d68a3128"}
Aug 02 10:38:13.874686 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:13.874343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z" event={"ID":"b61d89c3-54aa-4f47-aeb6-e45095114e7b","Type":"ContainerStarted","Data":"12059a63a4dcce4895301827ed1f75dd789bfff8b463dd0be641759ec24ca0e7"}
Aug 02 10:38:13.892082 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:13.892008    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-hhd8z" podStartSLOduration=1.506455469 podStartE2EDuration="2.891989136s" totalImagesPullingTime="1.385533667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:38:13.891826363 +0000 UTC m=+295.422409012" watchObservedRunningTime="2026-08-02 10:38:13.891989136 +0000 UTC m=+295.422571768"
Aug 02 10:38:14.844708 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:14.844673    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6hstf"
Aug 02 10:38:15.468321 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.468286    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7qz8m"]
Aug 02 10:38:15.474312 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.474282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.477516 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.477417    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 02 10:38:15.477516 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.477460    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 02 10:38:15.477516 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.477479    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 02 10:38:15.477516 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.477468    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7h556\""
Aug 02 10:38:15.653492 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.653459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12dbc719-95c7-43a9-aaf6-ed8b42db093f-sys\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.653492 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.653497    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12dbc719-95c7-43a9-aaf6-ed8b42db093f-metrics-client-ca\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.653692 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.653514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.653692 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.653589    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-textfile\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.653692 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.653612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12dbc719-95c7-43a9-aaf6-ed8b42db093f-root\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.653692 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.653650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82nn5\" (UniqueName: \"kubernetes.io/projected/12dbc719-95c7-43a9-aaf6-ed8b42db093f-kube-api-access-82nn5\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.653692 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.653668    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-wtmp\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.653692 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.653690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-accelerators-collector-config\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.653942 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.653716    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-tls\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754264 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754176    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-textfile\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754264 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754212    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12dbc719-95c7-43a9-aaf6-ed8b42db093f-root\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754264 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82nn5\" (UniqueName: \"kubernetes.io/projected/12dbc719-95c7-43a9-aaf6-ed8b42db093f-kube-api-access-82nn5\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754529 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-wtmp\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754529 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-accelerators-collector-config\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754529 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754337    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-tls\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754529 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12dbc719-95c7-43a9-aaf6-ed8b42db093f-sys\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754529 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12dbc719-95c7-43a9-aaf6-ed8b42db093f-root\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754529 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754479    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-wtmp\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754529 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12dbc719-95c7-43a9-aaf6-ed8b42db093f-metrics-client-ca\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754857 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754535    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754857 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754554    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-textfile\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754857 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12dbc719-95c7-43a9-aaf6-ed8b42db093f-sys\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.754992 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.754936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12dbc719-95c7-43a9-aaf6-ed8b42db093f-metrics-client-ca\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.755044 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.755010    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-accelerators-collector-config\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.756828 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.756806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-tls\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.756907 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.756883    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12dbc719-95c7-43a9-aaf6-ed8b42db093f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.762974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.762953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82nn5\" (UniqueName: \"kubernetes.io/projected/12dbc719-95c7-43a9-aaf6-ed8b42db093f-kube-api-access-82nn5\") pod \"node-exporter-7qz8m\" (UID: \"12dbc719-95c7-43a9-aaf6-ed8b42db093f\") " pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.785076 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.785055    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7qz8m"
Aug 02 10:38:15.798173 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:38:15.798149    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12dbc719_95c7_43a9_aaf6_ed8b42db093f.slice/crio-2337d551fe85456209dbe06a3965ca479d571ec4b13481bc9db20183af47a90d WatchSource:0}: Error finding container 2337d551fe85456209dbe06a3965ca479d571ec4b13481bc9db20183af47a90d: Status 404 returned error can't find the container with id 2337d551fe85456209dbe06a3965ca479d571ec4b13481bc9db20183af47a90d
Aug 02 10:38:15.882324 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:15.882283    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7qz8m" event={"ID":"12dbc719-95c7-43a9-aaf6-ed8b42db093f","Type":"ContainerStarted","Data":"2337d551fe85456209dbe06a3965ca479d571ec4b13481bc9db20183af47a90d"}
Aug 02 10:38:17.890791 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:17.890754    2576 generic.go:356] "Generic (PLEG): container finished" podID="12dbc719-95c7-43a9-aaf6-ed8b42db093f" containerID="1248371c20842537e30c1d60b2cd4a393402bcb183418cf0e312166234797a18" exitCode=0
Aug 02 10:38:17.891275 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:17.890825    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7qz8m" event={"ID":"12dbc719-95c7-43a9-aaf6-ed8b42db093f","Type":"ContainerDied","Data":"1248371c20842537e30c1d60b2cd4a393402bcb183418cf0e312166234797a18"}
Aug 02 10:38:18.511347 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.511300    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-64d7484547-hjx52"]
Aug 02 10:38:18.514997 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.514973    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.517722 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.517696    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Aug 02 10:38:18.517722 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.517712    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Aug 02 10:38:18.517903 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.517742    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bm872\""
Aug 02 10:38:18.517903 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.517696    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Aug 02 10:38:18.517903 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.517696    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4oig1ofj1abig\""
Aug 02 10:38:18.517903 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.517696    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Aug 02 10:38:18.518245 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.518227    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Aug 02 10:38:18.528522 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.528503    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64d7484547-hjx52"]
Aug 02 10:38:18.578867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.578834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.578867 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.578871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5bln\" (UniqueName: \"kubernetes.io/projected/826c0237-d299-46e3-b73d-2f7c0f3d7d15-kube-api-access-x5bln\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.579123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.578895    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.579123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.578960    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-tls\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.579123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.578998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/826c0237-d299-46e3-b73d-2f7c0f3d7d15-metrics-client-ca\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.579123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.579018    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.579123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.579067    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.579123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.579086    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-grpc-tls\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.679788 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.679752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/826c0237-d299-46e3-b73d-2f7c0f3d7d15-metrics-client-ca\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.679788 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.679791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.680024 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.679810    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.680024 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.679828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-grpc-tls\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.680024 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.679851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.680024 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.679883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5bln\" (UniqueName: \"kubernetes.io/projected/826c0237-d299-46e3-b73d-2f7c0f3d7d15-kube-api-access-x5bln\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.680024 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.679912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.680024 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.679962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-tls\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.680671 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.680610    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/826c0237-d299-46e3-b73d-2f7c0f3d7d15-metrics-client-ca\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.682682 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.682653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.682792 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.682683    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-tls\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.683204 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.683186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.683276 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.683216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.683276 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.683222    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.683669 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.683651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/826c0237-d299-46e3-b73d-2f7c0f3d7d15-secret-grpc-tls\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.689318 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.689296    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5bln\" (UniqueName: \"kubernetes.io/projected/826c0237-d299-46e3-b73d-2f7c0f3d7d15-kube-api-access-x5bln\") pod \"thanos-querier-64d7484547-hjx52\" (UID: \"826c0237-d299-46e3-b73d-2f7c0f3d7d15\") " pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.813661 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.813579    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:38:18.814162 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.814141    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:38:18.819474 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.819456    2576 kubelet.go:1648] "Image garbage collection succeeded"
Aug 02 10:38:18.826299 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.826278    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bm872\""
Aug 02 10:38:18.834354 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.834338    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:18.897821 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.897787    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7qz8m" event={"ID":"12dbc719-95c7-43a9-aaf6-ed8b42db093f","Type":"ContainerStarted","Data":"ee98bcef53231e3a069953e46af4883483cab5c5c58c4c160ec204a4b5a4f4a1"}
Aug 02 10:38:18.898294 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.897851    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7qz8m" event={"ID":"12dbc719-95c7-43a9-aaf6-ed8b42db093f","Type":"ContainerStarted","Data":"8d1190f28ea6660f9db54e56a241c0962845713ce19bd9a7b0c80f265ecc559a"}
Aug 02 10:38:18.931062 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.930962    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7qz8m" podStartSLOduration=2.812148288 podStartE2EDuration="3.93093893s" totalImagesPullingTime="1.0464029s" totalInitContainerRuntime="72.387742ms" isStatefulPod=true podCreationTimestamp="2026-08-02 10:38:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:38:18.928859631 +0000 UTC m=+300.459442284" watchObservedRunningTime="2026-08-02 10:38:18.93093893 +0000 UTC m=+300.461521562"
Aug 02 10:38:18.978542 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:18.978409    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64d7484547-hjx52"]
Aug 02 10:38:18.985697 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:38:18.981159    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod826c0237_d299_46e3_b73d_2f7c0f3d7d15.slice/crio-a07696dbab1070a387af86fff21c11dd6837f83d81f347b8abded00cc8816afa WatchSource:0}: Error finding container a07696dbab1070a387af86fff21c11dd6837f83d81f347b8abded00cc8816afa: Status 404 returned error can't find the container with id a07696dbab1070a387af86fff21c11dd6837f83d81f347b8abded00cc8816afa
Aug 02 10:38:19.903407 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:19.903369    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52" event={"ID":"826c0237-d299-46e3-b73d-2f7c0f3d7d15","Type":"ContainerStarted","Data":"a07696dbab1070a387af86fff21c11dd6837f83d81f347b8abded00cc8816afa"}
Aug 02 10:38:20.909699 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:20.909654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52" event={"ID":"826c0237-d299-46e3-b73d-2f7c0f3d7d15","Type":"ContainerStarted","Data":"8b7214f96891e7256e2788c92328e9df85f5e4f487eb3ccad9c94755d4679714"}
Aug 02 10:38:21.017667 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:21.017646    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 02 10:38:21.917666 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:21.917623    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52" event={"ID":"826c0237-d299-46e3-b73d-2f7c0f3d7d15","Type":"ContainerStarted","Data":"7bdba598bf0f7f16863c9bdb81ff7dfdfc6340e447e42f05e23c36f34a9c290a"}
Aug 02 10:38:21.918051 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:21.917693    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52" event={"ID":"826c0237-d299-46e3-b73d-2f7c0f3d7d15","Type":"ContainerStarted","Data":"0e6750f9f6ceb2a67fc21f76b1f9de152ba33467a6ff7ad65542e2d21bb8c069"}
Aug 02 10:38:21.918051 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:21.917724    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52" event={"ID":"826c0237-d299-46e3-b73d-2f7c0f3d7d15","Type":"ContainerStarted","Data":"ee918da7e94dbdebf441b76238bdfb9b4b9c8458e2fd4b6b4a50e1d81255053a"}
Aug 02 10:38:21.918051 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:21.917752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52" event={"ID":"826c0237-d299-46e3-b73d-2f7c0f3d7d15","Type":"ContainerStarted","Data":"337d50ad17084064463c5397ae12410f10854b9c875ec23dc3398727a704ccd0"}
Aug 02 10:38:21.918051 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:21.917795    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52" event={"ID":"826c0237-d299-46e3-b73d-2f7c0f3d7d15","Type":"ContainerStarted","Data":"47d1ecd363e361ad160c70f00c95751e3ed531ef00d7505cc69516506a64310f"}
Aug 02 10:38:21.918051 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:21.917833    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:21.943124 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:21.943073    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52" podStartSLOduration=1.476936996 podStartE2EDuration="3.943052125s" totalImagesPullingTime="2.466115129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:38:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:38:21.941651371 +0000 UTC m=+303.472234001" watchObservedRunningTime="2026-08-02 10:38:21.943052125 +0000 UTC m=+303.473634761"
Aug 02 10:38:23.944740 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:23.944703    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:38:27.866748 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:27.866679    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6df46b759b-kktpn" podUID="c27b5d62-9959-434c-b16f-edf16b7498e8" containerName="registry" containerID="cri-o://bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c" gracePeriod=30
Aug 02 10:38:27.928786 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:27.928763    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64d7484547-hjx52"
Aug 02 10:38:28.105584 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.105557    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:38:28.156132 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.156107    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-certificates\") pod \"c27b5d62-9959-434c-b16f-edf16b7498e8\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") "
Aug 02 10:38:28.156269 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.156149    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-installation-pull-secrets\") pod \"c27b5d62-9959-434c-b16f-edf16b7498e8\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") "
Aug 02 10:38:28.156269 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.156200    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-image-registry-private-configuration\") pod \"c27b5d62-9959-434c-b16f-edf16b7498e8\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") "
Aug 02 10:38:28.156269 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.156231    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") pod \"c27b5d62-9959-434c-b16f-edf16b7498e8\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") "
Aug 02 10:38:28.156269 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.156257    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c27b5d62-9959-434c-b16f-edf16b7498e8-ca-trust-extracted\") pod \"c27b5d62-9959-434c-b16f-edf16b7498e8\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") "
Aug 02 10:38:28.156460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.156284    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-trusted-ca\") pod \"c27b5d62-9959-434c-b16f-edf16b7498e8\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") "
Aug 02 10:38:28.156460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.156314    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-bound-sa-token\") pod \"c27b5d62-9959-434c-b16f-edf16b7498e8\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") "
Aug 02 10:38:28.156460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.156350    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvkzr\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-kube-api-access-hvkzr\") pod \"c27b5d62-9959-434c-b16f-edf16b7498e8\" (UID: \"c27b5d62-9959-434c-b16f-edf16b7498e8\") "
Aug 02 10:38:28.156602 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.156512    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c27b5d62-9959-434c-b16f-edf16b7498e8" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:38:28.157139 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.157100    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c27b5d62-9959-434c-b16f-edf16b7498e8" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:38:28.158890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.158867    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c27b5d62-9959-434c-b16f-edf16b7498e8" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:38:28.159126 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.159075    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c27b5d62-9959-434c-b16f-edf16b7498e8" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:38:28.159233 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.159164    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c27b5d62-9959-434c-b16f-edf16b7498e8" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:38:28.159233 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.159170    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-kube-api-access-hvkzr" (OuterVolumeSpecName: "kube-api-access-hvkzr") pod "c27b5d62-9959-434c-b16f-edf16b7498e8" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8"). InnerVolumeSpecName "kube-api-access-hvkzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:38:28.159344 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.159270    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c27b5d62-9959-434c-b16f-edf16b7498e8" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:38:28.164236 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.164214    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c27b5d62-9959-434c-b16f-edf16b7498e8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c27b5d62-9959-434c-b16f-edf16b7498e8" (UID: "c27b5d62-9959-434c-b16f-edf16b7498e8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:38:28.256865 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.256832    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvkzr\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-kube-api-access-hvkzr\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:38:28.256865 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.256860    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-certificates\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:38:28.256865 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.256871    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-installation-pull-secrets\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:38:28.257137 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.256881    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c27b5d62-9959-434c-b16f-edf16b7498e8-image-registry-private-configuration\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:38:28.257137 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.256891    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-registry-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:38:28.257137 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.256899    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c27b5d62-9959-434c-b16f-edf16b7498e8-ca-trust-extracted\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:38:28.257137 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.256907    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c27b5d62-9959-434c-b16f-edf16b7498e8-trusted-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:38:28.257137 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.256915    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c27b5d62-9959-434c-b16f-edf16b7498e8-bound-sa-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:38:28.941797 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.941761    2576 generic.go:356] "Generic (PLEG): container finished" podID="c27b5d62-9959-434c-b16f-edf16b7498e8" containerID="bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c" exitCode=0
Aug 02 10:38:28.942249 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.941822    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6df46b759b-kktpn"
Aug 02 10:38:28.942249 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.941862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6df46b759b-kktpn" event={"ID":"c27b5d62-9959-434c-b16f-edf16b7498e8","Type":"ContainerDied","Data":"bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c"}
Aug 02 10:38:28.942249 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.941939    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6df46b759b-kktpn" event={"ID":"c27b5d62-9959-434c-b16f-edf16b7498e8","Type":"ContainerDied","Data":"0e156856048806844e9af7940e89e33652d295e704456d1f49973e975903cbe8"}
Aug 02 10:38:28.942249 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.942020    2576 scope.go:117] "RemoveContainer" containerID="bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c"
Aug 02 10:38:28.953884 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.952758    2576 scope.go:117] "RemoveContainer" containerID="bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c"
Aug 02 10:38:28.953884 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:38:28.953144    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c\": container with ID starting with bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c not found: ID does not exist" containerID="bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c"
Aug 02 10:38:28.953884 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.953179    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c"} err="failed to get container status \"bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c\": rpc error: code = NotFound desc = could not find container \"bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c\": container with ID starting with bebf3d7aa928ec04968cdf8b00cac6f3c1c1f559b1cdb2177d8df9204cd8ce8c not found: ID does not exist"
Aug 02 10:38:28.969774 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.969750    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6df46b759b-kktpn"]
Aug 02 10:38:28.973312 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:28.973291    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6df46b759b-kktpn"]
Aug 02 10:38:30.949440 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:30.949408    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c27b5d62-9959-434c-b16f-edf16b7498e8" path="/var/lib/kubelet/pods/c27b5d62-9959-434c-b16f-edf16b7498e8/volumes"
Aug 02 10:38:32.856317 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.856280    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58f4959766-577tp"]
Aug 02 10:38:32.856780 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.856642    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c27b5d62-9959-434c-b16f-edf16b7498e8" containerName="registry"
Aug 02 10:38:32.856780 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.856659    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c27b5d62-9959-434c-b16f-edf16b7498e8" containerName="registry"
Aug 02 10:38:32.856780 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.856728    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c27b5d62-9959-434c-b16f-edf16b7498e8" containerName="registry"
Aug 02 10:38:32.861536 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.861515    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.864128 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.864102    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 02 10:38:32.864239 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.864161    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 02 10:38:32.865528 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.865391    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 02 10:38:32.865528 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.865401    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 02 10:38:32.865528 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.865443    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 02 10:38:32.865528 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.865454    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 02 10:38:32.865528 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.865522    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 02 10:38:32.865892 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.865877    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zzcmq\""
Aug 02 10:38:32.870066 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.870021    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f4959766-577tp"]
Aug 02 10:38:32.870651 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.870633    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Aug 02 10:38:32.891197 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.891169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-serving-cert\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.891331 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.891231    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-oauth-config\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.891331 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.891283    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t96zf\" (UniqueName: \"kubernetes.io/projected/53eed6fc-b874-44ff-ad95-fef4f407d67f-kube-api-access-t96zf\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.891451 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.891341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-oauth-serving-cert\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.891451 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.891378    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-config\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.891451 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.891404    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-trusted-ca-bundle\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.891451 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.891435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-service-ca\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.992454 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.992419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-config\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.992454 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.992455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-trusted-ca-bundle\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.992673 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.992481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-service-ca\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.992673 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.992543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-serving-cert\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.992673 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.992645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-oauth-config\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.992673 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.992667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t96zf\" (UniqueName: \"kubernetes.io/projected/53eed6fc-b874-44ff-ad95-fef4f407d67f-kube-api-access-t96zf\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.992882 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.992717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-oauth-serving-cert\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.993280 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.993252    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-service-ca\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.993564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.993304    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-config\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.993564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.993355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-oauth-serving-cert\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.993564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.993445    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-trusted-ca-bundle\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.995135 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.995109    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-serving-cert\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:32.995213 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:32.995169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-oauth-config\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:33.002102 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:33.002082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t96zf\" (UniqueName: \"kubernetes.io/projected/53eed6fc-b874-44ff-ad95-fef4f407d67f-kube-api-access-t96zf\") pod \"console-58f4959766-577tp\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") " pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:33.171721 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:33.171685    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:33.296984 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:33.296955    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f4959766-577tp"]
Aug 02 10:38:33.299890 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:38:33.299864    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53eed6fc_b874_44ff_ad95_fef4f407d67f.slice/crio-8711bac226d224be134fd75f69cde64650baa62591469ffa49b46aadced4aa5a WatchSource:0}: Error finding container 8711bac226d224be134fd75f69cde64650baa62591469ffa49b46aadced4aa5a: Status 404 returned error can't find the container with id 8711bac226d224be134fd75f69cde64650baa62591469ffa49b46aadced4aa5a
Aug 02 10:38:33.961479 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:33.961434    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f4959766-577tp" event={"ID":"53eed6fc-b874-44ff-ad95-fef4f407d67f","Type":"ContainerStarted","Data":"8711bac226d224be134fd75f69cde64650baa62591469ffa49b46aadced4aa5a"}
Aug 02 10:38:36.975531 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:36.975491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f4959766-577tp" event={"ID":"53eed6fc-b874-44ff-ad95-fef4f407d67f","Type":"ContainerStarted","Data":"1fe2995c0cfab1757ccdcf058bb40c0741932a221c938cbf8d02186a5dac1d4a"}
Aug 02 10:38:36.993599 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:36.993553    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58f4959766-577tp" podStartSLOduration=2.239779412 podStartE2EDuration="4.993537928s" totalImagesPullingTime="2.753758516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:38:36.992873297 +0000 UTC m=+318.523455925" watchObservedRunningTime="2026-08-02 10:38:36.993537928 +0000 UTC m=+318.524120619"
Aug 02 10:38:43.172347 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:43.172285    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:43.172731 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:43.172378    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:43.177118 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:43.177098    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:38:44.002189 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:38:44.002164    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:39:31.746271 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:31.746225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:39:31.748736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:31.748710    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f208a37-3b97-4072-ac20-9ffbf8732d84-metrics-certs\") pod \"network-metrics-daemon-mmf2n\" (UID: \"7f208a37-3b97-4072-ac20-9ffbf8732d84\") " pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:39:32.049199 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:32.049126    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dvcgg\""
Aug 02 10:39:32.056473 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:32.056455    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mmf2n"
Aug 02 10:39:32.181645 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:32.181605    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mmf2n"]
Aug 02 10:39:32.184387 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:39:32.184358    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f208a37_3b97_4072_ac20_9ffbf8732d84.slice/crio-87084b14e99ea8d39499f7d9643f444d0dadfe8e442a14372e7dabca501bbcef WatchSource:0}: Error finding container 87084b14e99ea8d39499f7d9643f444d0dadfe8e442a14372e7dabca501bbcef: Status 404 returned error can't find the container with id 87084b14e99ea8d39499f7d9643f444d0dadfe8e442a14372e7dabca501bbcef
Aug 02 10:39:33.169326 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:33.169298    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mmf2n" event={"ID":"7f208a37-3b97-4072-ac20-9ffbf8732d84","Type":"ContainerStarted","Data":"86aa32638a387b683766ffadf1814305d9f5c1210605d1d51b52699db50a4d1f"}
Aug 02 10:39:33.169618 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:33.169348    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mmf2n" event={"ID":"7f208a37-3b97-4072-ac20-9ffbf8732d84","Type":"ContainerStarted","Data":"87084b14e99ea8d39499f7d9643f444d0dadfe8e442a14372e7dabca501bbcef"}
Aug 02 10:39:34.174699 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:34.174655    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mmf2n" event={"ID":"7f208a37-3b97-4072-ac20-9ffbf8732d84","Type":"ContainerStarted","Data":"66630998d48fb83b6d5d79cce2b173fffc49828f32d4f4125b5a88abb03a0f50"}
Aug 02 10:39:34.193103 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:34.193057    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mmf2n" podStartSLOduration=375.3498601 podStartE2EDuration="6m16.193020709s" totalImagesPullingTime="843.160609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:39:34.191322049 +0000 UTC m=+375.721904677" watchObservedRunningTime="2026-08-02 10:39:34.193020709 +0000 UTC m=+375.723603356"
Aug 02 10:39:55.151405 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:39:55.151368    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f4959766-577tp"]
Aug 02 10:40:20.172072 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.171983    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58f4959766-577tp" podUID="53eed6fc-b874-44ff-ad95-fef4f407d67f" containerName="console" containerID="cri-o://1fe2995c0cfab1757ccdcf058bb40c0741932a221c938cbf8d02186a5dac1d4a" gracePeriod=15
Aug 02 10:40:20.333132 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.333094    2576 generic.go:356] "Generic (PLEG): container finished" podID="53eed6fc-b874-44ff-ad95-fef4f407d67f" containerID="1fe2995c0cfab1757ccdcf058bb40c0741932a221c938cbf8d02186a5dac1d4a" exitCode=0
Aug 02 10:40:20.333280 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.333145    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f4959766-577tp" event={"ID":"53eed6fc-b874-44ff-ad95-fef4f407d67f","Type":"ContainerDied","Data":"1fe2995c0cfab1757ccdcf058bb40c0741932a221c938cbf8d02186a5dac1d4a"}
Aug 02 10:40:20.411202 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.411178    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:40:20.509320 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509224    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-oauth-serving-cert\") pod \"53eed6fc-b874-44ff-ad95-fef4f407d67f\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") "
Aug 02 10:40:20.509320 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509281    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-trusted-ca-bundle\") pod \"53eed6fc-b874-44ff-ad95-fef4f407d67f\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") "
Aug 02 10:40:20.509320 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509299    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-service-ca\") pod \"53eed6fc-b874-44ff-ad95-fef4f407d67f\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") "
Aug 02 10:40:20.509598 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509413    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-serving-cert\") pod \"53eed6fc-b874-44ff-ad95-fef4f407d67f\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") "
Aug 02 10:40:20.509598 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509483    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t96zf\" (UniqueName: \"kubernetes.io/projected/53eed6fc-b874-44ff-ad95-fef4f407d67f-kube-api-access-t96zf\") pod \"53eed6fc-b874-44ff-ad95-fef4f407d67f\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") "
Aug 02 10:40:20.509598 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509509    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-config\") pod \"53eed6fc-b874-44ff-ad95-fef4f407d67f\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") "
Aug 02 10:40:20.509598 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509545    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-oauth-config\") pod \"53eed6fc-b874-44ff-ad95-fef4f407d67f\" (UID: \"53eed6fc-b874-44ff-ad95-fef4f407d67f\") "
Aug 02 10:40:20.509788 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509616    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "53eed6fc-b874-44ff-ad95-fef4f407d67f" (UID: "53eed6fc-b874-44ff-ad95-fef4f407d67f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:40:20.509788 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509644    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-service-ca" (OuterVolumeSpecName: "service-ca") pod "53eed6fc-b874-44ff-ad95-fef4f407d67f" (UID: "53eed6fc-b874-44ff-ad95-fef4f407d67f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:40:20.509788 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509690    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "53eed6fc-b874-44ff-ad95-fef4f407d67f" (UID: "53eed6fc-b874-44ff-ad95-fef4f407d67f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:40:20.509927 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509874    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-config" (OuterVolumeSpecName: "console-config") pod "53eed6fc-b874-44ff-ad95-fef4f407d67f" (UID: "53eed6fc-b874-44ff-ad95-fef4f407d67f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:40:20.509982 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509930    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-oauth-serving-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:40:20.509982 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509951    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-trusted-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:40:20.509982 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.509965    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:40:20.511761 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.511734    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "53eed6fc-b874-44ff-ad95-fef4f407d67f" (UID: "53eed6fc-b874-44ff-ad95-fef4f407d67f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:40:20.511889 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.511769    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "53eed6fc-b874-44ff-ad95-fef4f407d67f" (UID: "53eed6fc-b874-44ff-ad95-fef4f407d67f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:40:20.511889 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.511833    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53eed6fc-b874-44ff-ad95-fef4f407d67f-kube-api-access-t96zf" (OuterVolumeSpecName: "kube-api-access-t96zf") pod "53eed6fc-b874-44ff-ad95-fef4f407d67f" (UID: "53eed6fc-b874-44ff-ad95-fef4f407d67f"). InnerVolumeSpecName "kube-api-access-t96zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:40:20.610880 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.610851    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-serving-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:40:20.610880 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.610878    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t96zf\" (UniqueName: \"kubernetes.io/projected/53eed6fc-b874-44ff-ad95-fef4f407d67f-kube-api-access-t96zf\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:40:20.611083 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.610893    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:40:20.611083 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:20.610905    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/53eed6fc-b874-44ff-ad95-fef4f407d67f-console-oauth-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:40:21.337977 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:21.337898    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f4959766-577tp" event={"ID":"53eed6fc-b874-44ff-ad95-fef4f407d67f","Type":"ContainerDied","Data":"8711bac226d224be134fd75f69cde64650baa62591469ffa49b46aadced4aa5a"}
Aug 02 10:40:21.337977 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:21.337953    2576 scope.go:117] "RemoveContainer" containerID="1fe2995c0cfab1757ccdcf058bb40c0741932a221c938cbf8d02186a5dac1d4a"
Aug 02 10:40:21.338417 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:21.337906    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f4959766-577tp"
Aug 02 10:40:21.360178 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:21.360152    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f4959766-577tp"]
Aug 02 10:40:21.363606 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:21.363585    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58f4959766-577tp"]
Aug 02 10:40:22.949009 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:22.948973    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53eed6fc-b874-44ff-ad95-fef4f407d67f" path="/var/lib/kubelet/pods/53eed6fc-b874-44ff-ad95-fef4f407d67f/volumes"
Aug 02 10:40:53.063162 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.063085    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6666fbfdc5-9kxx6"]
Aug 02 10:40:53.063605 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.063361    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53eed6fc-b874-44ff-ad95-fef4f407d67f" containerName="console"
Aug 02 10:40:53.063605 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.063372    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="53eed6fc-b874-44ff-ad95-fef4f407d67f" containerName="console"
Aug 02 10:40:53.063605 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.063416    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="53eed6fc-b874-44ff-ad95-fef4f407d67f" containerName="console"
Aug 02 10:40:53.066117 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.066093    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.068877 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.068846    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 02 10:40:53.069004 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.068880    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 02 10:40:53.069004 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.068908    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 02 10:40:53.070326 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.070307    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 02 10:40:53.070326 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.070320    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 02 10:40:53.070466 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.070341    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 02 10:40:53.070466 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.070307    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 02 10:40:53.074154 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.070900    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zzcmq\""
Aug 02 10:40:53.079765 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.079743    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Aug 02 10:40:53.080056 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.080015    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6666fbfdc5-9kxx6"]
Aug 02 10:40:53.143305 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.143274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-service-ca\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.143460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.143338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-oauth-serving-cert\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.143460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.143365    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-serving-cert\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.143460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.143392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-trusted-ca-bundle\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.143460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.143423    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjg5x\" (UniqueName: \"kubernetes.io/projected/90f4762f-68de-43f8-bcc2-bdc6110960e2-kube-api-access-pjg5x\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.143460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.143444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-oauth-config\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.143638 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.143460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-config\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.244458 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.244412    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjg5x\" (UniqueName: \"kubernetes.io/projected/90f4762f-68de-43f8-bcc2-bdc6110960e2-kube-api-access-pjg5x\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.244634 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.244468    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-oauth-config\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.244634 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.244492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-config\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.244634 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.244531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-service-ca\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.244634 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.244572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-oauth-serving-cert\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.244634 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.244604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-serving-cert\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.244849 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.244635    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-trusted-ca-bundle\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.245455 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.245427    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-config\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.245566 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.245427    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-oauth-serving-cert\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.245566 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.245511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-service-ca\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.245888 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.245866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-trusted-ca-bundle\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.247218 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.247196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-serving-cert\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.247334 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.247319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-oauth-config\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.253991 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.253969    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjg5x\" (UniqueName: \"kubernetes.io/projected/90f4762f-68de-43f8-bcc2-bdc6110960e2-kube-api-access-pjg5x\") pod \"console-6666fbfdc5-9kxx6\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") " pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.379164 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.379077    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:40:53.509161 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:53.509130    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6666fbfdc5-9kxx6"]
Aug 02 10:40:53.511859 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:40:53.511830    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90f4762f_68de_43f8_bcc2_bdc6110960e2.slice/crio-c8a0e023ddeb9b0ab5deaf173f8f51765f3a580e482d115952a5df3fec8f967c WatchSource:0}: Error finding container c8a0e023ddeb9b0ab5deaf173f8f51765f3a580e482d115952a5df3fec8f967c: Status 404 returned error can't find the container with id c8a0e023ddeb9b0ab5deaf173f8f51765f3a580e482d115952a5df3fec8f967c
Aug 02 10:40:54.450758 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:54.450723    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6666fbfdc5-9kxx6" event={"ID":"90f4762f-68de-43f8-bcc2-bdc6110960e2","Type":"ContainerStarted","Data":"2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b"}
Aug 02 10:40:54.451147 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:54.450776    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6666fbfdc5-9kxx6" event={"ID":"90f4762f-68de-43f8-bcc2-bdc6110960e2","Type":"ContainerStarted","Data":"c8a0e023ddeb9b0ab5deaf173f8f51765f3a580e482d115952a5df3fec8f967c"}
Aug 02 10:40:54.471132 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:40:54.471052    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6666fbfdc5-9kxx6" podStartSLOduration=1.471017287 podStartE2EDuration="1.471017287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:40:54.470258502 +0000 UTC m=+456.000841155" watchObservedRunningTime="2026-08-02 10:40:54.471017287 +0000 UTC m=+456.001599916"
Aug 02 10:41:03.379461 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:03.379411    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:41:03.379949 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:03.379496    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:41:03.384437 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:03.384411    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:41:03.488771 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:03.488745    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:41:28.960582 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.960551    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"]
Aug 02 10:41:28.965128 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.965105    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:28.969157 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.969134    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Aug 02 10:41:28.969157 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.969149    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 02 10:41:28.969330 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.969190    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 02 10:41:28.969330 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.969214    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 02 10:41:28.976052 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.976008    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"]
Aug 02 10:41:28.992145 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.992117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/181113ea-d317-4556-b0d5-0e282f8304a8-tmp\") pod \"klusterlet-addon-workmgr-7b7d9c7779-br8kk\" (UID: \"181113ea-d317-4556-b0d5-0e282f8304a8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:28.992264 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.992246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6f6c\" (UniqueName: \"kubernetes.io/projected/181113ea-d317-4556-b0d5-0e282f8304a8-kube-api-access-q6f6c\") pod \"klusterlet-addon-workmgr-7b7d9c7779-br8kk\" (UID: \"181113ea-d317-4556-b0d5-0e282f8304a8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:28.992315 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:28.992287    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/181113ea-d317-4556-b0d5-0e282f8304a8-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b7d9c7779-br8kk\" (UID: \"181113ea-d317-4556-b0d5-0e282f8304a8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:29.093168 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:29.093131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/181113ea-d317-4556-b0d5-0e282f8304a8-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b7d9c7779-br8kk\" (UID: \"181113ea-d317-4556-b0d5-0e282f8304a8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:29.093340 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:29.093199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/181113ea-d317-4556-b0d5-0e282f8304a8-tmp\") pod \"klusterlet-addon-workmgr-7b7d9c7779-br8kk\" (UID: \"181113ea-d317-4556-b0d5-0e282f8304a8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:29.093340 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:29.093239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6f6c\" (UniqueName: \"kubernetes.io/projected/181113ea-d317-4556-b0d5-0e282f8304a8-kube-api-access-q6f6c\") pod \"klusterlet-addon-workmgr-7b7d9c7779-br8kk\" (UID: \"181113ea-d317-4556-b0d5-0e282f8304a8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:29.093568 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:29.093548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/181113ea-d317-4556-b0d5-0e282f8304a8-tmp\") pod \"klusterlet-addon-workmgr-7b7d9c7779-br8kk\" (UID: \"181113ea-d317-4556-b0d5-0e282f8304a8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:29.095612 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:29.095594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/181113ea-d317-4556-b0d5-0e282f8304a8-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b7d9c7779-br8kk\" (UID: \"181113ea-d317-4556-b0d5-0e282f8304a8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:29.101763 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:29.101732    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6f6c\" (UniqueName: \"kubernetes.io/projected/181113ea-d317-4556-b0d5-0e282f8304a8-kube-api-access-q6f6c\") pod \"klusterlet-addon-workmgr-7b7d9c7779-br8kk\" (UID: \"181113ea-d317-4556-b0d5-0e282f8304a8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:29.274162 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:29.274066    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:29.402878 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:29.402826    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"]
Aug 02 10:41:29.405366 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:41:29.405339    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod181113ea_d317_4556_b0d5_0e282f8304a8.slice/crio-3c9780049094bc279b8c0967d76225c460a6e84a131d1e65a0b8367816ce95bc WatchSource:0}: Error finding container 3c9780049094bc279b8c0967d76225c460a6e84a131d1e65a0b8367816ce95bc: Status 404 returned error can't find the container with id 3c9780049094bc279b8c0967d76225c460a6e84a131d1e65a0b8367816ce95bc
Aug 02 10:41:29.575344 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:29.575265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk" event={"ID":"181113ea-d317-4556-b0d5-0e282f8304a8","Type":"ContainerStarted","Data":"3c9780049094bc279b8c0967d76225c460a6e84a131d1e65a0b8367816ce95bc"}
Aug 02 10:41:33.597113 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:33.597076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk" event={"ID":"181113ea-d317-4556-b0d5-0e282f8304a8","Type":"ContainerStarted","Data":"2b771189ecae1bd7a5bdb3ec98c713971ccb229fc61e619bf39c96086c3052b5"}
Aug 02 10:41:33.597718 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:33.597290    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:33.599059 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:33.599020    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk"
Aug 02 10:41:33.616359 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:41:33.616311    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7d9c7779-br8kk" podStartSLOduration=1.9030881389999998 podStartE2EDuration="5.616298002s" totalImagesPullingTime="3.713209863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:41:33.614855749 +0000 UTC m=+495.145438389" watchObservedRunningTime="2026-08-02 10:41:33.616298002 +0000 UTC m=+495.146880630"
Aug 02 10:42:03.408041 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.407998    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"]
Aug 02 10:42:03.450893 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.450867    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"]
Aug 02 10:42:03.451079 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.450993    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.453809 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.453789    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 02 10:42:03.454817 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.454797    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fgtkk\""
Aug 02 10:42:03.454926 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.454802    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 02 10:42:03.535787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.535755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr8b4\" (UniqueName: \"kubernetes.io/projected/710ecc21-4f90-4436-8963-08c993c29f35-kube-api-access-xr8b4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.535919 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.535793    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.535919 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.535813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.636950 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.636915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr8b4\" (UniqueName: \"kubernetes.io/projected/710ecc21-4f90-4436-8963-08c993c29f35-kube-api-access-xr8b4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.637130 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.636971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.637130 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.636997    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.637374 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.637352    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.637442 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.637372    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.645769 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.645745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr8b4\" (UniqueName: \"kubernetes.io/projected/710ecc21-4f90-4436-8963-08c993c29f35-kube-api-access-xr8b4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.760283 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.760220    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:03.903840 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:03.903810    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"]
Aug 02 10:42:03.907323 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:42:03.907293    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod710ecc21_4f90_4436_8963_08c993c29f35.slice/crio-999ff8b0569293bb0ff4384687601a146f827013b7d67e49556901e7bb84ac73 WatchSource:0}: Error finding container 999ff8b0569293bb0ff4384687601a146f827013b7d67e49556901e7bb84ac73: Status 404 returned error can't find the container with id 999ff8b0569293bb0ff4384687601a146f827013b7d67e49556901e7bb84ac73
Aug 02 10:42:04.703467 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:04.703421    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t" event={"ID":"710ecc21-4f90-4436-8963-08c993c29f35","Type":"ContainerStarted","Data":"999ff8b0569293bb0ff4384687601a146f827013b7d67e49556901e7bb84ac73"}
Aug 02 10:42:10.726299 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:10.726262    2576 generic.go:356] "Generic (PLEG): container finished" podID="710ecc21-4f90-4436-8963-08c993c29f35" containerID="0e6d2932b51e54b3d091caba5ae93e75ff8a223a7b1e722f97862955b4451cc2" exitCode=0
Aug 02 10:42:10.726698 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:10.726325    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t" event={"ID":"710ecc21-4f90-4436-8963-08c993c29f35","Type":"ContainerDied","Data":"0e6d2932b51e54b3d091caba5ae93e75ff8a223a7b1e722f97862955b4451cc2"}
Aug 02 10:42:13.739234 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:13.739199    2576 generic.go:356] "Generic (PLEG): container finished" podID="710ecc21-4f90-4436-8963-08c993c29f35" containerID="a11c30f5d00bac1e55180864ead406e27640f03daed752234cc003bc09cbd3a5" exitCode=0
Aug 02 10:42:13.739828 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:13.739248    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t" event={"ID":"710ecc21-4f90-4436-8963-08c993c29f35","Type":"ContainerDied","Data":"a11c30f5d00bac1e55180864ead406e27640f03daed752234cc003bc09cbd3a5"}
Aug 02 10:42:20.772645 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:20.772611    2576 generic.go:356] "Generic (PLEG): container finished" podID="710ecc21-4f90-4436-8963-08c993c29f35" containerID="0ea297f48f40ed716fc5eaa60e90142f658950c58e1c933d637561d0e83b8aca" exitCode=0
Aug 02 10:42:20.773113 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:20.772701    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t" event={"ID":"710ecc21-4f90-4436-8963-08c993c29f35","Type":"ContainerDied","Data":"0ea297f48f40ed716fc5eaa60e90142f658950c58e1c933d637561d0e83b8aca"}
Aug 02 10:42:21.897891 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:21.897869    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:22.088762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.088690    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-bundle\") pod \"710ecc21-4f90-4436-8963-08c993c29f35\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") "
Aug 02 10:42:22.088762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.088752    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-util\") pod \"710ecc21-4f90-4436-8963-08c993c29f35\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") "
Aug 02 10:42:22.088930 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.088778    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr8b4\" (UniqueName: \"kubernetes.io/projected/710ecc21-4f90-4436-8963-08c993c29f35-kube-api-access-xr8b4\") pod \"710ecc21-4f90-4436-8963-08c993c29f35\" (UID: \"710ecc21-4f90-4436-8963-08c993c29f35\") "
Aug 02 10:42:22.089487 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.089418    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-bundle" (OuterVolumeSpecName: "bundle") pod "710ecc21-4f90-4436-8963-08c993c29f35" (UID: "710ecc21-4f90-4436-8963-08c993c29f35"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:42:22.091186 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.091164    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/710ecc21-4f90-4436-8963-08c993c29f35-kube-api-access-xr8b4" (OuterVolumeSpecName: "kube-api-access-xr8b4") pod "710ecc21-4f90-4436-8963-08c993c29f35" (UID: "710ecc21-4f90-4436-8963-08c993c29f35"). InnerVolumeSpecName "kube-api-access-xr8b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:42:22.093139 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.093116    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-util" (OuterVolumeSpecName: "util") pod "710ecc21-4f90-4436-8963-08c993c29f35" (UID: "710ecc21-4f90-4436-8963-08c993c29f35"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:42:22.189275 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.189231    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:42:22.189275 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.189268    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/710ecc21-4f90-4436-8963-08c993c29f35-util\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:42:22.189275 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.189278    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr8b4\" (UniqueName: \"kubernetes.io/projected/710ecc21-4f90-4436-8963-08c993c29f35-kube-api-access-xr8b4\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:42:22.780727 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.780693    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t" event={"ID":"710ecc21-4f90-4436-8963-08c993c29f35","Type":"ContainerDied","Data":"999ff8b0569293bb0ff4384687601a146f827013b7d67e49556901e7bb84ac73"}
Aug 02 10:42:22.780888 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.780741    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="999ff8b0569293bb0ff4384687601a146f827013b7d67e49556901e7bb84ac73"
Aug 02 10:42:22.780888 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:22.780741    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405bf5t"
Aug 02 10:42:26.568500 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.568468    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"]
Aug 02 10:42:26.568922 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.568860    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="710ecc21-4f90-4436-8963-08c993c29f35" containerName="extract"
Aug 02 10:42:26.568922 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.568875    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="710ecc21-4f90-4436-8963-08c993c29f35" containerName="extract"
Aug 02 10:42:26.568922 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.568898    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="710ecc21-4f90-4436-8963-08c993c29f35" containerName="util"
Aug 02 10:42:26.568922 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.568907    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="710ecc21-4f90-4436-8963-08c993c29f35" containerName="util"
Aug 02 10:42:26.568922 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.568916    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="710ecc21-4f90-4436-8963-08c993c29f35" containerName="pull"
Aug 02 10:42:26.568922 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.568924    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="710ecc21-4f90-4436-8963-08c993c29f35" containerName="pull"
Aug 02 10:42:26.569123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.568987    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="710ecc21-4f90-4436-8963-08c993c29f35" containerName="extract"
Aug 02 10:42:26.621173 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.621148    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"]
Aug 02 10:42:26.621302 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.621237    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"
Aug 02 10:42:26.624393 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.624372    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Aug 02 10:42:26.624494 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.624446    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Aug 02 10:42:26.624494 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.624453    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cj2bv\""
Aug 02 10:42:26.719268 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.719239    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6c298cd-db98-4ec9-99be-cd9d12c6f2de-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5z4bp\" (UID: \"d6c298cd-db98-4ec9-99be-cd9d12c6f2de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"
Aug 02 10:42:26.719386 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.719328    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6tkj\" (UniqueName: \"kubernetes.io/projected/d6c298cd-db98-4ec9-99be-cd9d12c6f2de-kube-api-access-g6tkj\") pod \"cert-manager-operator-controller-manager-55c444bf86-5z4bp\" (UID: \"d6c298cd-db98-4ec9-99be-cd9d12c6f2de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"
Aug 02 10:42:26.819697 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.819632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6tkj\" (UniqueName: \"kubernetes.io/projected/d6c298cd-db98-4ec9-99be-cd9d12c6f2de-kube-api-access-g6tkj\") pod \"cert-manager-operator-controller-manager-55c444bf86-5z4bp\" (UID: \"d6c298cd-db98-4ec9-99be-cd9d12c6f2de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"
Aug 02 10:42:26.819697 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.819680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6c298cd-db98-4ec9-99be-cd9d12c6f2de-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5z4bp\" (UID: \"d6c298cd-db98-4ec9-99be-cd9d12c6f2de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"
Aug 02 10:42:26.820010 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.819994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6c298cd-db98-4ec9-99be-cd9d12c6f2de-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5z4bp\" (UID: \"d6c298cd-db98-4ec9-99be-cd9d12c6f2de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"
Aug 02 10:42:26.828802 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.828768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6tkj\" (UniqueName: \"kubernetes.io/projected/d6c298cd-db98-4ec9-99be-cd9d12c6f2de-kube-api-access-g6tkj\") pod \"cert-manager-operator-controller-manager-55c444bf86-5z4bp\" (UID: \"d6c298cd-db98-4ec9-99be-cd9d12c6f2de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"
Aug 02 10:42:26.929974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:26.929945    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"
Aug 02 10:42:27.083513 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:27.083484    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp"]
Aug 02 10:42:27.086070 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:42:27.086040    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c298cd_db98_4ec9_99be_cd9d12c6f2de.slice/crio-dd7e216cf35ec9d48247a3da8616257fd44676b08054101e1c021cc9bd367e44 WatchSource:0}: Error finding container dd7e216cf35ec9d48247a3da8616257fd44676b08054101e1c021cc9bd367e44: Status 404 returned error can't find the container with id dd7e216cf35ec9d48247a3da8616257fd44676b08054101e1c021cc9bd367e44
Aug 02 10:42:27.800411 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:27.800381    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp" event={"ID":"d6c298cd-db98-4ec9-99be-cd9d12c6f2de","Type":"ContainerStarted","Data":"dd7e216cf35ec9d48247a3da8616257fd44676b08054101e1c021cc9bd367e44"}
Aug 02 10:42:29.811084 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:29.811041    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp" event={"ID":"d6c298cd-db98-4ec9-99be-cd9d12c6f2de","Type":"ContainerStarted","Data":"c5ae98c7faa6bfb876e31344fd83ee8f930e09eac97e475a6fb8750bffbf003d"}
Aug 02 10:42:29.833264 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:29.833210    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5z4bp" podStartSLOduration=1.570264428 podStartE2EDuration="3.833189428s" totalImagesPullingTime="2.262925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:42:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:42:29.832628958 +0000 UTC m=+551.363211589" watchObservedRunningTime="2026-08-02 10:42:29.833189428 +0000 UTC m=+551.363772058"
Aug 02 10:42:30.927634 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:30.927606    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"]
Aug 02 10:42:30.947930 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:30.947904    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:30.950959 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:30.950937    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 02 10:42:30.952248 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:30.952224    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 02 10:42:30.952373 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:30.952228    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fgtkk\""
Aug 02 10:42:30.954528 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:30.954507    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"]
Aug 02 10:42:31.046746 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.046716    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztm9v\" (UniqueName: \"kubernetes.io/projected/47d4854b-3f26-4b43-b153-6215f09c94f4-kube-api-access-ztm9v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.046895 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.046757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.046895 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.046861    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.147245 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.147209    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztm9v\" (UniqueName: \"kubernetes.io/projected/47d4854b-3f26-4b43-b153-6215f09c94f4-kube-api-access-ztm9v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.147245 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.147248    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.147425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.147290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.147737 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.147719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.147779 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.147737    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.157284 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.157262    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztm9v\" (UniqueName: \"kubernetes.io/projected/47d4854b-3f26-4b43-b153-6215f09c94f4-kube-api-access-ztm9v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.257658 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.257595    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:31.398395 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.398367    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"]
Aug 02 10:42:31.401334 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:42:31.401309    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d4854b_3f26_4b43_b153_6215f09c94f4.slice/crio-312ca3899f03e844e51f69c496c5b82b54b83ad2fb0b0ef31fae167410a1cc49 WatchSource:0}: Error finding container 312ca3899f03e844e51f69c496c5b82b54b83ad2fb0b0ef31fae167410a1cc49: Status 404 returned error can't find the container with id 312ca3899f03e844e51f69c496c5b82b54b83ad2fb0b0ef31fae167410a1cc49
Aug 02 10:42:31.820201 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.820111    2576 generic.go:356] "Generic (PLEG): container finished" podID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerID="dbec95577e1dc07dd5348d8577e6af15fe29cdaffe5571c5b3224599ef3faeb2" exitCode=0
Aug 02 10:42:31.820201 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.820184    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk" event={"ID":"47d4854b-3f26-4b43-b153-6215f09c94f4","Type":"ContainerDied","Data":"dbec95577e1dc07dd5348d8577e6af15fe29cdaffe5571c5b3224599ef3faeb2"}
Aug 02 10:42:31.820427 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:31.820229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk" event={"ID":"47d4854b-3f26-4b43-b153-6215f09c94f4","Type":"ContainerStarted","Data":"312ca3899f03e844e51f69c496c5b82b54b83ad2fb0b0ef31fae167410a1cc49"}
Aug 02 10:42:34.834736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:34.834704    2576 generic.go:356] "Generic (PLEG): container finished" podID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerID="5a19b1e5649565339b52c2c9ae1f48cc35a1909fcc0a925ef705190823c46e1b" exitCode=0
Aug 02 10:42:34.835155 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:34.834770    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk" event={"ID":"47d4854b-3f26-4b43-b153-6215f09c94f4","Type":"ContainerDied","Data":"5a19b1e5649565339b52c2c9ae1f48cc35a1909fcc0a925ef705190823c46e1b"}
Aug 02 10:42:35.841209 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:35.841176    2576 generic.go:356] "Generic (PLEG): container finished" podID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerID="960db387b19d7fb839d7bc4c27b1c5d3d9b56f34f34e7a69c0556e24e45eec1e" exitCode=0
Aug 02 10:42:35.841575 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:35.841238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk" event={"ID":"47d4854b-3f26-4b43-b153-6215f09c94f4","Type":"ContainerDied","Data":"960db387b19d7fb839d7bc4c27b1c5d3d9b56f34f34e7a69c0556e24e45eec1e"}
Aug 02 10:42:36.801280 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.801252    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6fh4b"]
Aug 02 10:42:36.803179 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.803162    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:36.805827 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.805808    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 02 10:42:36.805987 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.805968    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2jwkf\""
Aug 02 10:42:36.806063 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.806006    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 02 10:42:36.815590 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.815568    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6fh4b"]
Aug 02 10:42:36.883183 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.883152    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhjrc\" (UniqueName: \"kubernetes.io/projected/74d560f9-7a06-402c-b15c-b0739856fd46-kube-api-access-fhjrc\") pod \"cert-manager-webhook-6486ffd997-6fh4b\" (UID: \"74d560f9-7a06-402c-b15c-b0739856fd46\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:36.883542 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.883209    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74d560f9-7a06-402c-b15c-b0739856fd46-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6fh4b\" (UID: \"74d560f9-7a06-402c-b15c-b0739856fd46\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:36.970548 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.970524    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:36.984065 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.984021    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-bundle\") pod \"47d4854b-3f26-4b43-b153-6215f09c94f4\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") "
Aug 02 10:42:36.984187 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.984135    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjrc\" (UniqueName: \"kubernetes.io/projected/74d560f9-7a06-402c-b15c-b0739856fd46-kube-api-access-fhjrc\") pod \"cert-manager-webhook-6486ffd997-6fh4b\" (UID: \"74d560f9-7a06-402c-b15c-b0739856fd46\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:36.984257 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.984212    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74d560f9-7a06-402c-b15c-b0739856fd46-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6fh4b\" (UID: \"74d560f9-7a06-402c-b15c-b0739856fd46\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:36.984465 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.984419    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-bundle" (OuterVolumeSpecName: "bundle") pod "47d4854b-3f26-4b43-b153-6215f09c94f4" (UID: "47d4854b-3f26-4b43-b153-6215f09c94f4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:42:36.999621 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.999597    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhjrc\" (UniqueName: \"kubernetes.io/projected/74d560f9-7a06-402c-b15c-b0739856fd46-kube-api-access-fhjrc\") pod \"cert-manager-webhook-6486ffd997-6fh4b\" (UID: \"74d560f9-7a06-402c-b15c-b0739856fd46\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:36.999736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:36.999676    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74d560f9-7a06-402c-b15c-b0739856fd46-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6fh4b\" (UID: \"74d560f9-7a06-402c-b15c-b0739856fd46\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:37.085131 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.085050    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-util\") pod \"47d4854b-3f26-4b43-b153-6215f09c94f4\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") "
Aug 02 10:42:37.085131 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.085115    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztm9v\" (UniqueName: \"kubernetes.io/projected/47d4854b-3f26-4b43-b153-6215f09c94f4-kube-api-access-ztm9v\") pod \"47d4854b-3f26-4b43-b153-6215f09c94f4\" (UID: \"47d4854b-3f26-4b43-b153-6215f09c94f4\") "
Aug 02 10:42:37.085301 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.085240    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:42:37.087252 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.087212    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47d4854b-3f26-4b43-b153-6215f09c94f4-kube-api-access-ztm9v" (OuterVolumeSpecName: "kube-api-access-ztm9v") pod "47d4854b-3f26-4b43-b153-6215f09c94f4" (UID: "47d4854b-3f26-4b43-b153-6215f09c94f4"). InnerVolumeSpecName "kube-api-access-ztm9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:42:37.089420 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.089399    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-util" (OuterVolumeSpecName: "util") pod "47d4854b-3f26-4b43-b153-6215f09c94f4" (UID: "47d4854b-3f26-4b43-b153-6215f09c94f4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:42:37.122969 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.122935    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:37.186086 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.185998    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztm9v\" (UniqueName: \"kubernetes.io/projected/47d4854b-3f26-4b43-b153-6215f09c94f4-kube-api-access-ztm9v\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:42:37.186180 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.186090    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47d4854b-3f26-4b43-b153-6215f09c94f4-util\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:42:37.251309 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.251287    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6fh4b"]
Aug 02 10:42:37.253704 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:42:37.253683    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74d560f9_7a06_402c_b15c_b0739856fd46.slice/crio-b01677a85fd07c67416d4829a1c1cce325c59be4ae882b0805d02c1ec41b8dd0 WatchSource:0}: Error finding container b01677a85fd07c67416d4829a1c1cce325c59be4ae882b0805d02c1ec41b8dd0: Status 404 returned error can't find the container with id b01677a85fd07c67416d4829a1c1cce325c59be4ae882b0805d02c1ec41b8dd0
Aug 02 10:42:37.851460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.851424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b" event={"ID":"74d560f9-7a06-402c-b15c-b0739856fd46","Type":"ContainerStarted","Data":"b01677a85fd07c67416d4829a1c1cce325c59be4ae882b0805d02c1ec41b8dd0"}
Aug 02 10:42:37.853574 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.853547    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk" event={"ID":"47d4854b-3f26-4b43-b153-6215f09c94f4","Type":"ContainerDied","Data":"312ca3899f03e844e51f69c496c5b82b54b83ad2fb0b0ef31fae167410a1cc49"}
Aug 02 10:42:37.853704 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.853592    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnrjnk"
Aug 02 10:42:37.853704 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:37.853594    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="312ca3899f03e844e51f69c496c5b82b54b83ad2fb0b0ef31fae167410a1cc49"
Aug 02 10:42:39.863806 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:39.863714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b" event={"ID":"74d560f9-7a06-402c-b15c-b0739856fd46","Type":"ContainerStarted","Data":"5a39b47aa754860a1765e378aa6ef8c75e5adf30db268a40e52c4f6f62bc4981"}
Aug 02 10:42:39.864212 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:39.863856    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:39.882102 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:39.882060    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b" podStartSLOduration=1.5918207309999999 podStartE2EDuration="3.882014389s" totalImagesPullingTime="2.290193658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:42:39.881291621 +0000 UTC m=+561.411874249" watchObservedRunningTime="2026-08-02 10:42:39.882014389 +0000 UTC m=+561.412597019"
Aug 02 10:42:40.582554 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.582523    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8qrwp"]
Aug 02 10:42:40.582800 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.582789    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerName="extract"
Aug 02 10:42:40.582844 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.582801    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerName="extract"
Aug 02 10:42:40.582844 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.582817    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerName="pull"
Aug 02 10:42:40.582844 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.582823    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerName="pull"
Aug 02 10:42:40.582844 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.582836    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerName="util"
Aug 02 10:42:40.582844 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.582841    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerName="util"
Aug 02 10:42:40.582991 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.582884    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="47d4854b-3f26-4b43-b153-6215f09c94f4" containerName="extract"
Aug 02 10:42:40.585793 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.585777    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8qrwp"
Aug 02 10:42:40.588976 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.588953    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-q7ttv\""
Aug 02 10:42:40.597921 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.597897    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8qrwp"]
Aug 02 10:42:40.610840 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.610807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e78aa39-2196-4695-b289-ad241e881b68-bound-sa-token\") pod \"cert-manager-55b4d954d5-8qrwp\" (UID: \"0e78aa39-2196-4695-b289-ad241e881b68\") " pod="cert-manager/cert-manager-55b4d954d5-8qrwp"
Aug 02 10:42:40.610840 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.610839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfjml\" (UniqueName: \"kubernetes.io/projected/0e78aa39-2196-4695-b289-ad241e881b68-kube-api-access-hfjml\") pod \"cert-manager-55b4d954d5-8qrwp\" (UID: \"0e78aa39-2196-4695-b289-ad241e881b68\") " pod="cert-manager/cert-manager-55b4d954d5-8qrwp"
Aug 02 10:42:40.711949 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.711916    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e78aa39-2196-4695-b289-ad241e881b68-bound-sa-token\") pod \"cert-manager-55b4d954d5-8qrwp\" (UID: \"0e78aa39-2196-4695-b289-ad241e881b68\") " pod="cert-manager/cert-manager-55b4d954d5-8qrwp"
Aug 02 10:42:40.711949 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.711953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfjml\" (UniqueName: \"kubernetes.io/projected/0e78aa39-2196-4695-b289-ad241e881b68-kube-api-access-hfjml\") pod \"cert-manager-55b4d954d5-8qrwp\" (UID: \"0e78aa39-2196-4695-b289-ad241e881b68\") " pod="cert-manager/cert-manager-55b4d954d5-8qrwp"
Aug 02 10:42:40.722057 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.722001    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e78aa39-2196-4695-b289-ad241e881b68-bound-sa-token\") pod \"cert-manager-55b4d954d5-8qrwp\" (UID: \"0e78aa39-2196-4695-b289-ad241e881b68\") " pod="cert-manager/cert-manager-55b4d954d5-8qrwp"
Aug 02 10:42:40.722177 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.722158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfjml\" (UniqueName: \"kubernetes.io/projected/0e78aa39-2196-4695-b289-ad241e881b68-kube-api-access-hfjml\") pod \"cert-manager-55b4d954d5-8qrwp\" (UID: \"0e78aa39-2196-4695-b289-ad241e881b68\") " pod="cert-manager/cert-manager-55b4d954d5-8qrwp"
Aug 02 10:42:40.895113 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:40.895017    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8qrwp"
Aug 02 10:42:41.026520 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:41.026492    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8qrwp"]
Aug 02 10:42:41.029099 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:42:41.029073    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e78aa39_2196_4695_b289_ad241e881b68.slice/crio-64bd784cfc1ebbe66c506d04fd10286ef2e5affc3db7bdd415f266a92806998b WatchSource:0}: Error finding container 64bd784cfc1ebbe66c506d04fd10286ef2e5affc3db7bdd415f266a92806998b: Status 404 returned error can't find the container with id 64bd784cfc1ebbe66c506d04fd10286ef2e5affc3db7bdd415f266a92806998b
Aug 02 10:42:41.873315 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:41.873276    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8qrwp" event={"ID":"0e78aa39-2196-4695-b289-ad241e881b68","Type":"ContainerStarted","Data":"fb1401f2aba69fb97f4c996c80f640cac1a247ebef9e374902f9c034caaf964b"}
Aug 02 10:42:41.873315 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:41.873317    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8qrwp" event={"ID":"0e78aa39-2196-4695-b289-ad241e881b68","Type":"ContainerStarted","Data":"64bd784cfc1ebbe66c506d04fd10286ef2e5affc3db7bdd415f266a92806998b"}
Aug 02 10:42:41.890413 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:41.890369    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-8qrwp" podStartSLOduration=1.890352881 podStartE2EDuration="1.890352881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:42:41.889343792 +0000 UTC m=+563.419926424" watchObservedRunningTime="2026-08-02 10:42:41.890352881 +0000 UTC m=+563.420935509"
Aug 02 10:42:45.870354 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:45.870325    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6fh4b"
Aug 02 10:42:49.534120 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.534079    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"]
Aug 02 10:42:49.537303 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.537281    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.539992 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.539973    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 02 10:42:49.541156 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.541140    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 02 10:42:49.541251 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.541142    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fgtkk\""
Aug 02 10:42:49.545859 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.545837    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"]
Aug 02 10:42:49.579261 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.579233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.579370 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.579278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.579370 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.579301    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6km9m\" (UniqueName: \"kubernetes.io/projected/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-kube-api-access-6km9m\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.680010 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.679979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.680127 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.680040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.680127 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.680065    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6km9m\" (UniqueName: \"kubernetes.io/projected/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-kube-api-access-6km9m\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.680408 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.680392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.680443 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.680411    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.689065 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.689023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6km9m\" (UniqueName: \"kubernetes.io/projected/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-kube-api-access-6km9m\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.846884 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.846812    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:49.974999 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:49.974971    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"]
Aug 02 10:42:49.978858 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:42:49.978824    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba10d5e0_a9b7_4d20_b7e5_69e752987a77.slice/crio-be5d81efc02b6d0dce1d2c36ccbb6d0d46c49f1b61d9d53dbb6c4eab465d8c71 WatchSource:0}: Error finding container be5d81efc02b6d0dce1d2c36ccbb6d0d46c49f1b61d9d53dbb6c4eab465d8c71: Status 404 returned error can't find the container with id be5d81efc02b6d0dce1d2c36ccbb6d0d46c49f1b61d9d53dbb6c4eab465d8c71
Aug 02 10:42:50.914099 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:50.914066    2576 generic.go:356] "Generic (PLEG): container finished" podID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerID="30f41ce5f8d9ab0d5eb413482c817ae6817da1d186306cadbfbe1294a2572d7a" exitCode=0
Aug 02 10:42:50.914506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:50.914123    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q" event={"ID":"ba10d5e0-a9b7-4d20-b7e5-69e752987a77","Type":"ContainerDied","Data":"30f41ce5f8d9ab0d5eb413482c817ae6817da1d186306cadbfbe1294a2572d7a"}
Aug 02 10:42:50.914506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:50.914158    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q" event={"ID":"ba10d5e0-a9b7-4d20-b7e5-69e752987a77","Type":"ContainerStarted","Data":"be5d81efc02b6d0dce1d2c36ccbb6d0d46c49f1b61d9d53dbb6c4eab465d8c71"}
Aug 02 10:42:51.920132 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:51.920098    2576 generic.go:356] "Generic (PLEG): container finished" podID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerID="6ebf239193df7f544f6dea2c83d2bcd0dcdcc5357277f891dab066d9f041fbcd" exitCode=0
Aug 02 10:42:51.920444 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:51.920183    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q" event={"ID":"ba10d5e0-a9b7-4d20-b7e5-69e752987a77","Type":"ContainerDied","Data":"6ebf239193df7f544f6dea2c83d2bcd0dcdcc5357277f891dab066d9f041fbcd"}
Aug 02 10:42:52.927163 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:52.927133    2576 generic.go:356] "Generic (PLEG): container finished" podID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerID="83b1893d245b681873b89d3ca07aa4e5a36624c8e3620ecf8398ad389e494ca0" exitCode=0
Aug 02 10:42:52.927585 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:52.927215    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q" event={"ID":"ba10d5e0-a9b7-4d20-b7e5-69e752987a77","Type":"ContainerDied","Data":"83b1893d245b681873b89d3ca07aa4e5a36624c8e3620ecf8398ad389e494ca0"}
Aug 02 10:42:54.056889 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.056868    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:54.114550 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.114519    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6km9m\" (UniqueName: \"kubernetes.io/projected/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-kube-api-access-6km9m\") pod \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") "
Aug 02 10:42:54.114677 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.114574    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-bundle\") pod \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") "
Aug 02 10:42:54.114677 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.114589    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-util\") pod \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\" (UID: \"ba10d5e0-a9b7-4d20-b7e5-69e752987a77\") "
Aug 02 10:42:54.115229 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.115201    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-bundle" (OuterVolumeSpecName: "bundle") pod "ba10d5e0-a9b7-4d20-b7e5-69e752987a77" (UID: "ba10d5e0-a9b7-4d20-b7e5-69e752987a77"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:42:54.116664 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.116631    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-kube-api-access-6km9m" (OuterVolumeSpecName: "kube-api-access-6km9m") pod "ba10d5e0-a9b7-4d20-b7e5-69e752987a77" (UID: "ba10d5e0-a9b7-4d20-b7e5-69e752987a77"). InnerVolumeSpecName "kube-api-access-6km9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:42:54.120204 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.120180    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-util" (OuterVolumeSpecName: "util") pod "ba10d5e0-a9b7-4d20-b7e5-69e752987a77" (UID: "ba10d5e0-a9b7-4d20-b7e5-69e752987a77"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:42:54.215930 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.215862    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6km9m\" (UniqueName: \"kubernetes.io/projected/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-kube-api-access-6km9m\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:42:54.215930 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.215896    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:42:54.215930 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.215906    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba10d5e0-a9b7-4d20-b7e5-69e752987a77-util\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:42:54.938208 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.938181    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q"
Aug 02 10:42:54.938348 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.938177    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsd96q" event={"ID":"ba10d5e0-a9b7-4d20-b7e5-69e752987a77","Type":"ContainerDied","Data":"be5d81efc02b6d0dce1d2c36ccbb6d0d46c49f1b61d9d53dbb6c4eab465d8c71"}
Aug 02 10:42:54.938348 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:42:54.938299    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be5d81efc02b6d0dce1d2c36ccbb6d0d46c49f1b61d9d53dbb6c4eab465d8c71"
Aug 02 10:43:02.542901 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.542869    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"]
Aug 02 10:43:02.543502 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.543150    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerName="extract"
Aug 02 10:43:02.543502 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.543161    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerName="extract"
Aug 02 10:43:02.543502 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.543175    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerName="util"
Aug 02 10:43:02.543502 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.543180    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerName="util"
Aug 02 10:43:02.543502 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.543189    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerName="pull"
Aug 02 10:43:02.543502 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.543195    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerName="pull"
Aug 02 10:43:02.543502 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.543242    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba10d5e0-a9b7-4d20-b7e5-69e752987a77" containerName="extract"
Aug 02 10:43:02.547470 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.547453    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.550216 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.550200    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 02 10:43:02.550697 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.550681    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 02 10:43:02.550743 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.550719    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-64kwn\""
Aug 02 10:43:02.550786 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.550720    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 02 10:43:02.550868 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.550846    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 02 10:43:02.565989 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.565964    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"]
Aug 02 10:43:02.678911 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.678884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5687q\" (UniqueName: \"kubernetes.io/projected/558eb470-b32a-49a3-84c3-0d8b0d88a55b-kube-api-access-5687q\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.679069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.678918    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/558eb470-b32a-49a3-84c3-0d8b0d88a55b-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.679069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.678949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/558eb470-b32a-49a3-84c3-0d8b0d88a55b-tmp\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.679069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.679020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/558eb470-b32a-49a3-84c3-0d8b0d88a55b-manifests\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.679174 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.679106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/558eb470-b32a-49a3-84c3-0d8b0d88a55b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.779741 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.779705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/558eb470-b32a-49a3-84c3-0d8b0d88a55b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.779899 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.779753    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5687q\" (UniqueName: \"kubernetes.io/projected/558eb470-b32a-49a3-84c3-0d8b0d88a55b-kube-api-access-5687q\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.779899 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.779782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/558eb470-b32a-49a3-84c3-0d8b0d88a55b-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.779899 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.779822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/558eb470-b32a-49a3-84c3-0d8b0d88a55b-tmp\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.779899 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.779853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/558eb470-b32a-49a3-84c3-0d8b0d88a55b-manifests\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.780273 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.780253    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/558eb470-b32a-49a3-84c3-0d8b0d88a55b-manifests\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.782292 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.782264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/558eb470-b32a-49a3-84c3-0d8b0d88a55b-tmp\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.782467 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.782448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/558eb470-b32a-49a3-84c3-0d8b0d88a55b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.782510 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.782448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/558eb470-b32a-49a3-84c3-0d8b0d88a55b-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.789096 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.789075    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5687q\" (UniqueName: \"kubernetes.io/projected/558eb470-b32a-49a3-84c3-0d8b0d88a55b-kube-api-access-5687q\") pod \"opendatahub-operator-controller-manager-7f6ffbd656-wxq55\" (UID: \"558eb470-b32a-49a3-84c3-0d8b0d88a55b\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.858181 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.858116    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:02.998973 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:02.998945    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"]
Aug 02 10:43:03.001702 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:43:03.001677    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558eb470_b32a_49a3_84c3_0d8b0d88a55b.slice/crio-b78209efc5490f113003deb847dba3264bf2341add02a3e3a93435d2a7d0d7a3 WatchSource:0}: Error finding container b78209efc5490f113003deb847dba3264bf2341add02a3e3a93435d2a7d0d7a3: Status 404 returned error can't find the container with id b78209efc5490f113003deb847dba3264bf2341add02a3e3a93435d2a7d0d7a3
Aug 02 10:43:03.975571 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:03.975482    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55" event={"ID":"558eb470-b32a-49a3-84c3-0d8b0d88a55b","Type":"ContainerStarted","Data":"b78209efc5490f113003deb847dba3264bf2341add02a3e3a93435d2a7d0d7a3"}
Aug 02 10:43:05.985442 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:05.985417    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55" event={"ID":"558eb470-b32a-49a3-84c3-0d8b0d88a55b","Type":"ContainerStarted","Data":"ffe8733b15f16e73253d0cca608237241e2a7eb83455c0b1572b52f200ea1950"}
Aug 02 10:43:07.994125 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:07.994091    2576 generic.go:356] "Generic (PLEG): container finished" podID="558eb470-b32a-49a3-84c3-0d8b0d88a55b" containerID="ffe8733b15f16e73253d0cca608237241e2a7eb83455c0b1572b52f200ea1950" exitCode=0
Aug 02 10:43:07.994491 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:07.994162    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55" event={"ID":"558eb470-b32a-49a3-84c3-0d8b0d88a55b","Type":"ContainerDied","Data":"ffe8733b15f16e73253d0cca608237241e2a7eb83455c0b1572b52f200ea1950"}
Aug 02 10:43:09.000139 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:09.000108    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55" event={"ID":"558eb470-b32a-49a3-84c3-0d8b0d88a55b","Type":"ContainerStarted","Data":"6ae051c7e26e135c9c7ac9c63d243c7291b8e1637ba67d89600521658ecb4bd9"}
Aug 02 10:43:09.000563 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:09.000419    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:09.028153 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:09.025493    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55" podStartSLOduration=2.716564509 podStartE2EDuration="7.025473181s" totalImagesPullingTime="3.263774646s" totalInitContainerRuntime="1.045134026s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:43:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:43:09.022662807 +0000 UTC m=+590.553245435" watchObservedRunningTime="2026-08-02 10:43:09.025473181 +0000 UTC m=+590.556055813"
Aug 02 10:43:18.842280 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:18.842250    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:43:18.842813 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:18.842795    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:43:20.006517 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:20.006484    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f6ffbd656-wxq55"
Aug 02 10:43:25.749936 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.749903    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d654958d9-4k68m"]
Aug 02 10:43:25.753308 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.753286    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.767147 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.767126    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d654958d9-4k68m"]
Aug 02 10:43:25.859737 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.859705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-console-oauth-config\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.859910 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.859743    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-console-config\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.859910 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.859759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-oauth-serving-cert\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.859910 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.859782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nwzt\" (UniqueName: \"kubernetes.io/projected/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-kube-api-access-8nwzt\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.859910 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.859799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-trusted-ca-bundle\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.859910 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.859838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-console-serving-cert\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.859910 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.859875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-service-ca\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.960813 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.960776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nwzt\" (UniqueName: \"kubernetes.io/projected/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-kube-api-access-8nwzt\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.960813 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.960816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-trusted-ca-bundle\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.961107 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.960858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-console-serving-cert\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.961107 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.960874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-service-ca\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.961107 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.960912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-console-oauth-config\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.961107 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.960933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-console-config\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.961107 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.960952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-oauth-serving-cert\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.961693 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.961663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-console-config\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.961950 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.961928    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-oauth-serving-cert\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.961991 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.961947    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-trusted-ca-bundle\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.962061 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.961998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-service-ca\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.963633 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.963607    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-console-serving-cert\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.963728 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.963708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-console-oauth-config\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:25.969589 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:25.969560    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nwzt\" (UniqueName: \"kubernetes.io/projected/787b9dfa-1cf1-4ec5-80fd-b63982742c1e-kube-api-access-8nwzt\") pod \"console-7d654958d9-4k68m\" (UID: \"787b9dfa-1cf1-4ec5-80fd-b63982742c1e\") " pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:26.062477 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:26.062409    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:26.196643 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:26.196512    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d654958d9-4k68m"]
Aug 02 10:43:26.199475 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:43:26.199449    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod787b9dfa_1cf1_4ec5_80fd_b63982742c1e.slice/crio-56dcfeddbcd3579e10fe490d49d5e363cdd86ae7ddb5d433b11b3673bb8b0ec5 WatchSource:0}: Error finding container 56dcfeddbcd3579e10fe490d49d5e363cdd86ae7ddb5d433b11b3673bb8b0ec5: Status 404 returned error can't find the container with id 56dcfeddbcd3579e10fe490d49d5e363cdd86ae7ddb5d433b11b3673bb8b0ec5
Aug 02 10:43:27.088890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:27.088852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d654958d9-4k68m" event={"ID":"787b9dfa-1cf1-4ec5-80fd-b63982742c1e","Type":"ContainerStarted","Data":"ed10f7d81156fba0fd9356613f7ccf634f5911b95cef693039aea1cbac05a504"}
Aug 02 10:43:27.089391 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:27.088935    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d654958d9-4k68m" event={"ID":"787b9dfa-1cf1-4ec5-80fd-b63982742c1e","Type":"ContainerStarted","Data":"56dcfeddbcd3579e10fe490d49d5e363cdd86ae7ddb5d433b11b3673bb8b0ec5"}
Aug 02 10:43:27.110636 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:27.110592    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d654958d9-4k68m" podStartSLOduration=2.110578791 podStartE2EDuration="2.110578791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:43:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:43:27.109794717 +0000 UTC m=+608.640377357" watchObservedRunningTime="2026-08-02 10:43:27.110578791 +0000 UTC m=+608.641161419"
Aug 02 10:43:32.508225 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.508187    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"]
Aug 02 10:43:32.512576 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.512549    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.515372 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.515343    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 02 10:43:32.515475 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.515396    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j96wz\""
Aug 02 10:43:32.516581 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.516568    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 02 10:43:32.531817 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.531796    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"]
Aug 02 10:43:32.612393 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.612353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f3f2d6-cd3d-488a-93f8-3161686612f9-tls-certs\") pod \"kube-auth-proxy-7c47d49594-kbvx6\" (UID: \"48f3f2d6-cd3d-488a-93f8-3161686612f9\") " pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.612393 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.612390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48f3f2d6-cd3d-488a-93f8-3161686612f9-tmp\") pod \"kube-auth-proxy-7c47d49594-kbvx6\" (UID: \"48f3f2d6-cd3d-488a-93f8-3161686612f9\") " pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.612577 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.612494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wnvv\" (UniqueName: \"kubernetes.io/projected/48f3f2d6-cd3d-488a-93f8-3161686612f9-kube-api-access-6wnvv\") pod \"kube-auth-proxy-7c47d49594-kbvx6\" (UID: \"48f3f2d6-cd3d-488a-93f8-3161686612f9\") " pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.666463 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.666436    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kjwhq"]
Aug 02 10:43:32.673297 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.673241    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:32.676399 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.676364    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Aug 02 10:43:32.676745 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.676722    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-b595n\""
Aug 02 10:43:32.678526 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.678504    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kjwhq"]
Aug 02 10:43:32.713622 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.713596    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wnvv\" (UniqueName: \"kubernetes.io/projected/48f3f2d6-cd3d-488a-93f8-3161686612f9-kube-api-access-6wnvv\") pod \"kube-auth-proxy-7c47d49594-kbvx6\" (UID: \"48f3f2d6-cd3d-488a-93f8-3161686612f9\") " pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.713779 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.713644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f3f2d6-cd3d-488a-93f8-3161686612f9-tls-certs\") pod \"kube-auth-proxy-7c47d49594-kbvx6\" (UID: \"48f3f2d6-cd3d-488a-93f8-3161686612f9\") " pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.713779 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.713670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48f3f2d6-cd3d-488a-93f8-3161686612f9-tmp\") pod \"kube-auth-proxy-7c47d49594-kbvx6\" (UID: \"48f3f2d6-cd3d-488a-93f8-3161686612f9\") " pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.716152 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.716123    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48f3f2d6-cd3d-488a-93f8-3161686612f9-tmp\") pod \"kube-auth-proxy-7c47d49594-kbvx6\" (UID: \"48f3f2d6-cd3d-488a-93f8-3161686612f9\") " pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.716347 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.716328    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f3f2d6-cd3d-488a-93f8-3161686612f9-tls-certs\") pod \"kube-auth-proxy-7c47d49594-kbvx6\" (UID: \"48f3f2d6-cd3d-488a-93f8-3161686612f9\") " pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.722566 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.722536    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wnvv\" (UniqueName: \"kubernetes.io/projected/48f3f2d6-cd3d-488a-93f8-3161686612f9-kube-api-access-6wnvv\") pod \"kube-auth-proxy-7c47d49594-kbvx6\" (UID: \"48f3f2d6-cd3d-488a-93f8-3161686612f9\") " pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.815058 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.814959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntjvl\" (UniqueName: \"kubernetes.io/projected/c524b7ee-dc69-4b6c-8b06-deacc3c269e8-kube-api-access-ntjvl\") pod \"odh-model-controller-7998c67dc4-kjwhq\" (UID: \"c524b7ee-dc69-4b6c-8b06-deacc3c269e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:32.815190 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.815078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c524b7ee-dc69-4b6c-8b06-deacc3c269e8-cert\") pod \"odh-model-controller-7998c67dc4-kjwhq\" (UID: \"c524b7ee-dc69-4b6c-8b06-deacc3c269e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:32.822909 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.822892    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"
Aug 02 10:43:32.915669 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.915634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c524b7ee-dc69-4b6c-8b06-deacc3c269e8-cert\") pod \"odh-model-controller-7998c67dc4-kjwhq\" (UID: \"c524b7ee-dc69-4b6c-8b06-deacc3c269e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:32.915814 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.915724    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntjvl\" (UniqueName: \"kubernetes.io/projected/c524b7ee-dc69-4b6c-8b06-deacc3c269e8-kube-api-access-ntjvl\") pod \"odh-model-controller-7998c67dc4-kjwhq\" (UID: \"c524b7ee-dc69-4b6c-8b06-deacc3c269e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:32.916321 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:43:32.915984    2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Aug 02 10:43:32.916321 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:43:32.916074    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c524b7ee-dc69-4b6c-8b06-deacc3c269e8-cert podName:c524b7ee-dc69-4b6c-8b06-deacc3c269e8 nodeName:}" failed. No retries permitted until 2026-08-02 10:43:33.416051983 +0000 UTC m=+614.946634592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c524b7ee-dc69-4b6c-8b06-deacc3c269e8-cert") pod "odh-model-controller-7998c67dc4-kjwhq" (UID: "c524b7ee-dc69-4b6c-8b06-deacc3c269e8") : secret "odh-model-controller-webhook-cert" not found
Aug 02 10:43:32.925437 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.925402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntjvl\" (UniqueName: \"kubernetes.io/projected/c524b7ee-dc69-4b6c-8b06-deacc3c269e8-kube-api-access-ntjvl\") pod \"odh-model-controller-7998c67dc4-kjwhq\" (UID: \"c524b7ee-dc69-4b6c-8b06-deacc3c269e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:32.961378 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.961345    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6"]
Aug 02 10:43:32.964845 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:43:32.964823    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48f3f2d6_cd3d_488a_93f8_3161686612f9.slice/crio-8a999082a93ac3d4f737cabce5b3c5d9ef5e7f962f40d09f6f3a7f1e49de217c WatchSource:0}: Error finding container 8a999082a93ac3d4f737cabce5b3c5d9ef5e7f962f40d09f6f3a7f1e49de217c: Status 404 returned error can't find the container with id 8a999082a93ac3d4f737cabce5b3c5d9ef5e7f962f40d09f6f3a7f1e49de217c
Aug 02 10:43:32.966535 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:32.966522    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 02 10:43:33.115286 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.115207    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6" event={"ID":"48f3f2d6-cd3d-488a-93f8-3161686612f9","Type":"ContainerStarted","Data":"8a999082a93ac3d4f737cabce5b3c5d9ef5e7f962f40d09f6f3a7f1e49de217c"}
Aug 02 10:43:33.420557 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.420523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c524b7ee-dc69-4b6c-8b06-deacc3c269e8-cert\") pod \"odh-model-controller-7998c67dc4-kjwhq\" (UID: \"c524b7ee-dc69-4b6c-8b06-deacc3c269e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:33.423012 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.422990    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c524b7ee-dc69-4b6c-8b06-deacc3c269e8-cert\") pod \"odh-model-controller-7998c67dc4-kjwhq\" (UID: \"c524b7ee-dc69-4b6c-8b06-deacc3c269e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:33.586469 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.586433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:33.755126 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.755047    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kjwhq"]
Aug 02 10:43:33.757488 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:43:33.757456    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc524b7ee_dc69_4b6c_8b06_deacc3c269e8.slice/crio-187eaa0ed3ac8b4f95792ea09d9f17997ee23245fd37c92e319eaed6b41d7f9d WatchSource:0}: Error finding container 187eaa0ed3ac8b4f95792ea09d9f17997ee23245fd37c92e319eaed6b41d7f9d: Status 404 returned error can't find the container with id 187eaa0ed3ac8b4f95792ea09d9f17997ee23245fd37c92e319eaed6b41d7f9d
Aug 02 10:43:33.876845 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.876805    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"]
Aug 02 10:43:33.881585 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.881563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:33.884328 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.884293    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Aug 02 10:43:33.884468 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.884383    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Aug 02 10:43:33.884705 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.884687    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Aug 02 10:43:33.885071 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.885052    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 02 10:43:33.885344 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.885323    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v9mrl\""
Aug 02 10:43:33.900445 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:33.900380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"]
Aug 02 10:43:34.030059 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.029953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5kqz\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-kube-api-access-z5kqz\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.030059 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.030023    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04b3889-b298-4c18-94b4-c16813854a88-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.030298 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.030088    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.030298 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.030161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.030298 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.030226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.030298 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.030256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.030298 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.030282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.122638 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.122596    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq" event={"ID":"c524b7ee-dc69-4b6c-8b06-deacc3c269e8","Type":"ContainerStarted","Data":"187eaa0ed3ac8b4f95792ea09d9f17997ee23245fd37c92e319eaed6b41d7f9d"}
Aug 02 10:43:34.131109 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.131070    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04b3889-b298-4c18-94b4-c16813854a88-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.131257 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.131136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.131257 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.131187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.131378 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.131254    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.131378 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.131285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.131378 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.131313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.131378 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.131345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5kqz\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-kube-api-access-z5kqz\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.132168 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.132121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.135208 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.135184    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04b3889-b298-4c18-94b4-c16813854a88-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.135328 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.135216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.135599 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.135580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.136295 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.136272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.141149 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.141125    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.141890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.141849    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5kqz\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-kube-api-access-z5kqz\") pod \"istiod-openshift-gateway-7b7995d5cd-v68z6\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.194717 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.194680    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:34.363914 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:34.363883    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"]
Aug 02 10:43:35.228497 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:43:35.228457    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04b3889_b298_4c18_94b4_c16813854a88.slice/crio-45f4b3d3d6f959b4b11ed671da2a9e0f952f2e839458d533ddb87b91bcfbde7f WatchSource:0}: Error finding container 45f4b3d3d6f959b4b11ed671da2a9e0f952f2e839458d533ddb87b91bcfbde7f: Status 404 returned error can't find the container with id 45f4b3d3d6f959b4b11ed671da2a9e0f952f2e839458d533ddb87b91bcfbde7f
Aug 02 10:43:36.063782 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:36.063751    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:36.063980 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:36.063854    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:36.069202 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:36.069180    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:36.134159 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:36.134124    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6" event={"ID":"e04b3889-b298-4c18-94b4-c16813854a88","Type":"ContainerStarted","Data":"45f4b3d3d6f959b4b11ed671da2a9e0f952f2e839458d533ddb87b91bcfbde7f"}
Aug 02 10:43:36.138195 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:36.138176    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d654958d9-4k68m"
Aug 02 10:43:36.230991 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:36.230961    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6666fbfdc5-9kxx6"]
Aug 02 10:43:38.148315 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:38.146470    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq" event={"ID":"c524b7ee-dc69-4b6c-8b06-deacc3c269e8","Type":"ContainerStarted","Data":"277a5ae2871123bbece8c61a7e528f0abdf58164a7b1f2c25d3d1252691162de"}
Aug 02 10:43:38.148315 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:38.146569    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:43:38.149541 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:38.149198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6" event={"ID":"48f3f2d6-cd3d-488a-93f8-3161686612f9","Type":"ContainerStarted","Data":"539057c44815068e105304c4710937d0c38453b5f43294e8ab9a148e27425927"}
Aug 02 10:43:38.166636 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:38.166581    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq" podStartSLOduration=2.730756018 podStartE2EDuration="6.166566165s" totalImagesPullingTime="3.435810147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:43:38.165953583 +0000 UTC m=+619.696536223" watchObservedRunningTime="2026-08-02 10:43:38.166566165 +0000 UTC m=+619.697148793"
Aug 02 10:43:38.186765 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:38.186710    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c47d49594-kbvx6" podStartSLOduration=2.01243864 podStartE2EDuration="6.186689443s" totalImagesPullingTime="4.174250803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:43:38.185023107 +0000 UTC m=+619.715605755" watchObservedRunningTime="2026-08-02 10:43:38.186689443 +0000 UTC m=+619.717272073"
Aug 02 10:43:39.002935 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:39.002902    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 10:43:39.003013 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:39.002967    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 10:43:39.156583 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:39.156548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6" event={"ID":"e04b3889-b298-4c18-94b4-c16813854a88","Type":"ContainerStarted","Data":"4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0"}
Aug 02 10:43:39.157060 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:39.156877    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:39.182114 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:39.182065    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6" podStartSLOduration=2.410033752 podStartE2EDuration="6.18205024s" totalImagesPullingTime="3.772016488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:43:39.18060498 +0000 UTC m=+620.711187635" watchObservedRunningTime="2026-08-02 10:43:39.18205024 +0000 UTC m=+620.712632859"
Aug 02 10:43:40.162744 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:40.162665    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-v68z6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 02 10:43:40.162744 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:40.162727    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6" podUID="e04b3889-b298-4c18-94b4-c16813854a88" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 02 10:43:43.162718 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.162686    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:43.757911 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.757878    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"]
Aug 02 10:43:43.761058 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.761021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:43.764647 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.764619    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jqhgs\""
Aug 02 10:43:43.764771 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.764676    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Aug 02 10:43:43.782907 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.782880    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"]
Aug 02 10:43:43.805183 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.805158    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"]
Aug 02 10:43:43.809251 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.809227    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:43:43.813530 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.813508    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-74w4c\""
Aug 02 10:43:43.813685 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.813670    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Aug 02 10:43:43.818467 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.818447    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrlsb\" (UniqueName: \"kubernetes.io/projected/95918fe0-635a-4dc2-b18b-755b09b7def8-kube-api-access-hrlsb\") pod \"kserve-controller-manager-d6f4b65b8-zjsm4\" (UID: \"95918fe0-635a-4dc2-b18b-755b09b7def8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:43.818587 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.818569    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95918fe0-635a-4dc2-b18b-755b09b7def8-cert\") pod \"kserve-controller-manager-d6f4b65b8-zjsm4\" (UID: \"95918fe0-635a-4dc2-b18b-755b09b7def8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:43.818653 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.818607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e36da917-d50e-437a-b1f7-39b9b85ab068-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r7smk\" (UID: \"e36da917-d50e-437a-b1f7-39b9b85ab068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:43:43.818653 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.818640    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-959mh\" (UniqueName: \"kubernetes.io/projected/e36da917-d50e-437a-b1f7-39b9b85ab068-kube-api-access-959mh\") pod \"llmisvc-controller-manager-777c6bd7d5-r7smk\" (UID: \"e36da917-d50e-437a-b1f7-39b9b85ab068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:43:43.825580 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.825558    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"]
Aug 02 10:43:43.919108 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.919083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95918fe0-635a-4dc2-b18b-755b09b7def8-cert\") pod \"kserve-controller-manager-d6f4b65b8-zjsm4\" (UID: \"95918fe0-635a-4dc2-b18b-755b09b7def8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:43.919231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.919114    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e36da917-d50e-437a-b1f7-39b9b85ab068-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r7smk\" (UID: \"e36da917-d50e-437a-b1f7-39b9b85ab068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:43:43.919231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.919144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-959mh\" (UniqueName: \"kubernetes.io/projected/e36da917-d50e-437a-b1f7-39b9b85ab068-kube-api-access-959mh\") pod \"llmisvc-controller-manager-777c6bd7d5-r7smk\" (UID: \"e36da917-d50e-437a-b1f7-39b9b85ab068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:43:43.919231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.919187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrlsb\" (UniqueName: \"kubernetes.io/projected/95918fe0-635a-4dc2-b18b-755b09b7def8-kube-api-access-hrlsb\") pod \"kserve-controller-manager-d6f4b65b8-zjsm4\" (UID: \"95918fe0-635a-4dc2-b18b-755b09b7def8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:43.919352 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:43:43.919232    2576 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Aug 02 10:43:43.919352 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:43:43.919300    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95918fe0-635a-4dc2-b18b-755b09b7def8-cert podName:95918fe0-635a-4dc2-b18b-755b09b7def8 nodeName:}" failed. No retries permitted until 2026-08-02 10:43:44.419278227 +0000 UTC m=+625.949860836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/95918fe0-635a-4dc2-b18b-755b09b7def8-cert") pod "kserve-controller-manager-d6f4b65b8-zjsm4" (UID: "95918fe0-635a-4dc2-b18b-755b09b7def8") : secret "kserve-webhook-server-cert" not found
Aug 02 10:43:43.921707 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.921679    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e36da917-d50e-437a-b1f7-39b9b85ab068-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r7smk\" (UID: \"e36da917-d50e-437a-b1f7-39b9b85ab068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:43:43.930151 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.930120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrlsb\" (UniqueName: \"kubernetes.io/projected/95918fe0-635a-4dc2-b18b-755b09b7def8-kube-api-access-hrlsb\") pod \"kserve-controller-manager-d6f4b65b8-zjsm4\" (UID: \"95918fe0-635a-4dc2-b18b-755b09b7def8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:43.931064 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:43.931046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-959mh\" (UniqueName: \"kubernetes.io/projected/e36da917-d50e-437a-b1f7-39b9b85ab068-kube-api-access-959mh\") pod \"llmisvc-controller-manager-777c6bd7d5-r7smk\" (UID: \"e36da917-d50e-437a-b1f7-39b9b85ab068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:43:44.119552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:44.119458    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:43:44.267072 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:44.266977    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"]
Aug 02 10:43:44.269599 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:43:44.269574    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode36da917_d50e_437a_b1f7_39b9b85ab068.slice/crio-db86036d4c89604eed18606abe0200bf114c6d0cfa999f2edd22a4b5221e72bd WatchSource:0}: Error finding container db86036d4c89604eed18606abe0200bf114c6d0cfa999f2edd22a4b5221e72bd: Status 404 returned error can't find the container with id db86036d4c89604eed18606abe0200bf114c6d0cfa999f2edd22a4b5221e72bd
Aug 02 10:43:44.422741 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:44.422710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95918fe0-635a-4dc2-b18b-755b09b7def8-cert\") pod \"kserve-controller-manager-d6f4b65b8-zjsm4\" (UID: \"95918fe0-635a-4dc2-b18b-755b09b7def8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:44.425256 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:44.425236    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95918fe0-635a-4dc2-b18b-755b09b7def8-cert\") pod \"kserve-controller-manager-d6f4b65b8-zjsm4\" (UID: \"95918fe0-635a-4dc2-b18b-755b09b7def8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:44.671191 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:44.671158    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:44.830577 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:44.830548    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"]
Aug 02 10:43:44.875906 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:43:44.875858    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95918fe0_635a_4dc2_b18b_755b09b7def8.slice/crio-3d566f8453c90af6227c34cb58efdc6527cda87713fb1dcd64b09a3cb437cfab WatchSource:0}: Error finding container 3d566f8453c90af6227c34cb58efdc6527cda87713fb1dcd64b09a3cb437cfab: Status 404 returned error can't find the container with id 3d566f8453c90af6227c34cb58efdc6527cda87713fb1dcd64b09a3cb437cfab
Aug 02 10:43:45.184759 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.184728    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4" event={"ID":"95918fe0-635a-4dc2-b18b-755b09b7def8","Type":"ContainerStarted","Data":"3d566f8453c90af6227c34cb58efdc6527cda87713fb1dcd64b09a3cb437cfab"}
Aug 02 10:43:45.185742 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.185713    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk" event={"ID":"e36da917-d50e-437a-b1f7-39b9b85ab068","Type":"ContainerStarted","Data":"db86036d4c89604eed18606abe0200bf114c6d0cfa999f2edd22a4b5221e72bd"}
Aug 02 10:43:45.254173 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.254140    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"]
Aug 02 10:43:45.254350 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.254329    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6" podUID="e04b3889-b298-4c18-94b4-c16813854a88" containerName="discovery" containerID="cri-o://4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0" gracePeriod=30
Aug 02 10:43:45.572523 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.572476    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:45.632097 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.631738    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04b3889-b298-4c18-94b4-c16813854a88-local-certs\") pod \"e04b3889-b298-4c18-94b4-c16813854a88\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") "
Aug 02 10:43:45.632097 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.631774    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-kubeconfig\") pod \"e04b3889-b298-4c18-94b4-c16813854a88\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") "
Aug 02 10:43:45.632097 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.631798    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-istio-token\") pod \"e04b3889-b298-4c18-94b4-c16813854a88\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") "
Aug 02 10:43:45.632097 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.631839    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5kqz\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-kube-api-access-z5kqz\") pod \"e04b3889-b298-4c18-94b4-c16813854a88\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") "
Aug 02 10:43:45.632097 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.631869    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-cacerts\") pod \"e04b3889-b298-4c18-94b4-c16813854a88\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") "
Aug 02 10:43:45.632097 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.631923    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-dns-cert\") pod \"e04b3889-b298-4c18-94b4-c16813854a88\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") "
Aug 02 10:43:45.636871 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.636804    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e04b3889-b298-4c18-94b4-c16813854a88-local-certs" (OuterVolumeSpecName: "local-certs") pod "e04b3889-b298-4c18-94b4-c16813854a88" (UID: "e04b3889-b298-4c18-94b4-c16813854a88"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:43:45.636871 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.636824    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-kube-api-access-z5kqz" (OuterVolumeSpecName: "kube-api-access-z5kqz") pod "e04b3889-b298-4c18-94b4-c16813854a88" (UID: "e04b3889-b298-4c18-94b4-c16813854a88"). InnerVolumeSpecName "kube-api-access-z5kqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:43:45.636871 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.636814    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-cacerts" (OuterVolumeSpecName: "cacerts") pod "e04b3889-b298-4c18-94b4-c16813854a88" (UID: "e04b3889-b298-4c18-94b4-c16813854a88"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:43:45.636871 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.636870    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-istio-token" (OuterVolumeSpecName: "istio-token") pod "e04b3889-b298-4c18-94b4-c16813854a88" (UID: "e04b3889-b298-4c18-94b4-c16813854a88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:43:45.637425 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.637386    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e04b3889-b298-4c18-94b4-c16813854a88" (UID: "e04b3889-b298-4c18-94b4-c16813854a88"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:43:45.638422 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.638397    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e04b3889-b298-4c18-94b4-c16813854a88" (UID: "e04b3889-b298-4c18-94b4-c16813854a88"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:43:45.733926 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.733841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-ca-configmap\") pod \"e04b3889-b298-4c18-94b4-c16813854a88\" (UID: \"e04b3889-b298-4c18-94b4-c16813854a88\") "
Aug 02 10:43:45.734146 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.734127    2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-kubeconfig\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:43:45.734231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.734152    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-istio-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:43:45.734231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.734167    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5kqz\" (UniqueName: \"kubernetes.io/projected/e04b3889-b298-4c18-94b4-c16813854a88-kube-api-access-z5kqz\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:43:45.734231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.734183    2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-cacerts\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:43:45.734231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.734195    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-dns-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:43:45.734231 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.734209    2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04b3889-b298-4c18-94b4-c16813854a88-local-certs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:43:45.734470 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.734288    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e04b3889-b298-4c18-94b4-c16813854a88" (UID: "e04b3889-b298-4c18-94b4-c16813854a88"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:43:45.834792 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:45.834750    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e04b3889-b298-4c18-94b4-c16813854a88-istio-csr-ca-configmap\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:43:46.193249 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.193211    2576 generic.go:356] "Generic (PLEG): container finished" podID="e04b3889-b298-4c18-94b4-c16813854a88" containerID="4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0" exitCode=0
Aug 02 10:43:46.193422 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.193262    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"
Aug 02 10:43:46.193422 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.193296    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6" event={"ID":"e04b3889-b298-4c18-94b4-c16813854a88","Type":"ContainerDied","Data":"4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0"}
Aug 02 10:43:46.193422 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.193361    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6" event={"ID":"e04b3889-b298-4c18-94b4-c16813854a88","Type":"ContainerDied","Data":"45f4b3d3d6f959b4b11ed671da2a9e0f952f2e839458d533ddb87b91bcfbde7f"}
Aug 02 10:43:46.193422 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.193392    2576 scope.go:117] "RemoveContainer" containerID="4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0"
Aug 02 10:43:46.205047 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.204906    2576 scope.go:117] "RemoveContainer" containerID="4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0"
Aug 02 10:43:46.205227 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:43:46.205200    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0\": container with ID starting with 4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0 not found: ID does not exist" containerID="4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0"
Aug 02 10:43:46.205309 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.205238    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0"} err="failed to get container status \"4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0\": rpc error: code = NotFound desc = could not find container \"4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0\": container with ID starting with 4cf032044be48f35ce84730816bc586ea05c34c31300d75cc741ed5219e679d0 not found: ID does not exist"
Aug 02 10:43:46.221518 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.221492    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"]
Aug 02 10:43:46.233641 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.233605    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v68z6"]
Aug 02 10:43:46.950742 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:46.950703    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e04b3889-b298-4c18-94b4-c16813854a88" path="/var/lib/kubelet/pods/e04b3889-b298-4c18-94b4-c16813854a88/volumes"
Aug 02 10:43:48.205536 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:48.205502    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4" event={"ID":"95918fe0-635a-4dc2-b18b-755b09b7def8","Type":"ContainerStarted","Data":"3529133bad2131dec52a62f163b8b2d16892a4feab015eb2296061d90a57f29e"}
Aug 02 10:43:48.206011 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:48.205590    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:43:48.209724 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:48.209698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk" event={"ID":"e36da917-d50e-437a-b1f7-39b9b85ab068","Type":"ContainerStarted","Data":"6d2e75890419f85842c72979a3fc620db49d3d16b23afab1f91e517e3e72be97"}
Aug 02 10:43:48.209846 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:48.209820    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:43:48.238094 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:48.238046    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4" podStartSLOduration=2.638699038 podStartE2EDuration="5.238009885s" totalImagesPullingTime="2.599310847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:43:48.237416199 +0000 UTC m=+629.767998868" watchObservedRunningTime="2026-08-02 10:43:48.238009885 +0000 UTC m=+629.768592496"
Aug 02 10:43:48.264704 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:48.264652    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk" podStartSLOduration=2.061017224 podStartE2EDuration="5.26463268s" totalImagesPullingTime="3.203615456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:43:48.264229302 +0000 UTC m=+629.794811946" watchObservedRunningTime="2026-08-02 10:43:48.26463268 +0000 UTC m=+629.795215500"
Aug 02 10:43:49.159197 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:43:49.159170    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kjwhq"
Aug 02 10:44:01.251196 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.251146    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6666fbfdc5-9kxx6" podUID="90f4762f-68de-43f8-bcc2-bdc6110960e2" containerName="console" containerID="cri-o://2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b" gracePeriod=15
Aug 02 10:44:01.503375 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.503322    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:44:01.563902 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.563869    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-trusted-ca-bundle\") pod \"90f4762f-68de-43f8-bcc2-bdc6110960e2\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") "
Aug 02 10:44:01.563902 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.563905    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjg5x\" (UniqueName: \"kubernetes.io/projected/90f4762f-68de-43f8-bcc2-bdc6110960e2-kube-api-access-pjg5x\") pod \"90f4762f-68de-43f8-bcc2-bdc6110960e2\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") "
Aug 02 10:44:01.564130 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.563944    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-oauth-config\") pod \"90f4762f-68de-43f8-bcc2-bdc6110960e2\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") "
Aug 02 10:44:01.564130 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.564082    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-oauth-serving-cert\") pod \"90f4762f-68de-43f8-bcc2-bdc6110960e2\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") "
Aug 02 10:44:01.564130 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.564110    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-config\") pod \"90f4762f-68de-43f8-bcc2-bdc6110960e2\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") "
Aug 02 10:44:01.564291 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.564142    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-service-ca\") pod \"90f4762f-68de-43f8-bcc2-bdc6110960e2\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") "
Aug 02 10:44:01.564291 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.564206    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-serving-cert\") pod \"90f4762f-68de-43f8-bcc2-bdc6110960e2\" (UID: \"90f4762f-68de-43f8-bcc2-bdc6110960e2\") "
Aug 02 10:44:01.564447 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.564402    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "90f4762f-68de-43f8-bcc2-bdc6110960e2" (UID: "90f4762f-68de-43f8-bcc2-bdc6110960e2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:44:01.564534 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.564450    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "90f4762f-68de-43f8-bcc2-bdc6110960e2" (UID: "90f4762f-68de-43f8-bcc2-bdc6110960e2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:44:01.564534 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.564493    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-config" (OuterVolumeSpecName: "console-config") pod "90f4762f-68de-43f8-bcc2-bdc6110960e2" (UID: "90f4762f-68de-43f8-bcc2-bdc6110960e2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:44:01.564652 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.564559    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-service-ca" (OuterVolumeSpecName: "service-ca") pod "90f4762f-68de-43f8-bcc2-bdc6110960e2" (UID: "90f4762f-68de-43f8-bcc2-bdc6110960e2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:44:01.566341 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.566320    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "90f4762f-68de-43f8-bcc2-bdc6110960e2" (UID: "90f4762f-68de-43f8-bcc2-bdc6110960e2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:44:01.566341 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.566325    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90f4762f-68de-43f8-bcc2-bdc6110960e2-kube-api-access-pjg5x" (OuterVolumeSpecName: "kube-api-access-pjg5x") pod "90f4762f-68de-43f8-bcc2-bdc6110960e2" (UID: "90f4762f-68de-43f8-bcc2-bdc6110960e2"). InnerVolumeSpecName "kube-api-access-pjg5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:44:01.566484 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.566469    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "90f4762f-68de-43f8-bcc2-bdc6110960e2" (UID: "90f4762f-68de-43f8-bcc2-bdc6110960e2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:44:01.665069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.665012    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-serving-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:01.665069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.665064    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-trusted-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:01.665069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.665075    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjg5x\" (UniqueName: \"kubernetes.io/projected/90f4762f-68de-43f8-bcc2-bdc6110960e2-kube-api-access-pjg5x\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:01.665282 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.665084    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-oauth-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:01.665282 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.665093    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-oauth-serving-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:01.665282 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.665101    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-console-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:01.665282 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:01.665109    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90f4762f-68de-43f8-bcc2-bdc6110960e2-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:02.275276 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.275235    2576 generic.go:356] "Generic (PLEG): container finished" podID="90f4762f-68de-43f8-bcc2-bdc6110960e2" containerID="2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b" exitCode=0
Aug 02 10:44:02.275762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.275284    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6666fbfdc5-9kxx6"
Aug 02 10:44:02.275762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.275311    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6666fbfdc5-9kxx6" event={"ID":"90f4762f-68de-43f8-bcc2-bdc6110960e2","Type":"ContainerDied","Data":"2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b"}
Aug 02 10:44:02.275762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.275357    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6666fbfdc5-9kxx6" event={"ID":"90f4762f-68de-43f8-bcc2-bdc6110960e2","Type":"ContainerDied","Data":"c8a0e023ddeb9b0ab5deaf173f8f51765f3a580e482d115952a5df3fec8f967c"}
Aug 02 10:44:02.275762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.275376    2576 scope.go:117] "RemoveContainer" containerID="2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b"
Aug 02 10:44:02.283896 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.283881    2576 scope.go:117] "RemoveContainer" containerID="2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b"
Aug 02 10:44:02.284173 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:02.284148    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b\": container with ID starting with 2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b not found: ID does not exist" containerID="2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b"
Aug 02 10:44:02.284264 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.284180    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b"} err="failed to get container status \"2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b\": rpc error: code = NotFound desc = could not find container \"2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b\": container with ID starting with 2bf7018681b64f231c33c06f64388bed002679162a3ef598c6b9155d998e7d7b not found: ID does not exist"
Aug 02 10:44:02.297320 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.297299    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6666fbfdc5-9kxx6"]
Aug 02 10:44:02.300683 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.300661    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6666fbfdc5-9kxx6"]
Aug 02 10:44:02.949404 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:02.949368    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90f4762f-68de-43f8-bcc2-bdc6110960e2" path="/var/lib/kubelet/pods/90f4762f-68de-43f8-bcc2-bdc6110960e2/volumes"
Aug 02 10:44:19.218165 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:19.218137    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7smk"
Aug 02 10:44:19.221392 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:19.221376    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zjsm4"
Aug 02 10:44:26.935292 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.935249    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"]
Aug 02 10:44:26.935875 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.935701    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90f4762f-68de-43f8-bcc2-bdc6110960e2" containerName="console"
Aug 02 10:44:26.935875 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.935721    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="90f4762f-68de-43f8-bcc2-bdc6110960e2" containerName="console"
Aug 02 10:44:26.935875 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.935732    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e04b3889-b298-4c18-94b4-c16813854a88" containerName="discovery"
Aug 02 10:44:26.935875 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.935741    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e04b3889-b298-4c18-94b4-c16813854a88" containerName="discovery"
Aug 02 10:44:26.935875 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.935830    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="90f4762f-68de-43f8-bcc2-bdc6110960e2" containerName="console"
Aug 02 10:44:26.935875 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.935842    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e04b3889-b298-4c18-94b4-c16813854a88" containerName="discovery"
Aug 02 10:44:26.940689 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.940667    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:26.943639 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.943613    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 02 10:44:26.943762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.943739    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fgtkk\""
Aug 02 10:44:26.943762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.943749    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 02 10:44:26.957941 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:26.957914    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"]
Aug 02 10:44:27.025554 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.025524    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"]
Aug 02 10:44:27.028066 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.028048    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.038898 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.038877    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"]
Aug 02 10:44:27.084378 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.084342    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.084539 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.084380    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.084539 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.084410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whg8x\" (UniqueName: \"kubernetes.io/projected/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-kube-api-access-whg8x\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.125204 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.125164    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"]
Aug 02 10:44:27.127917 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.127895    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.139508 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.139488    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"]
Aug 02 10:44:27.185859 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.185795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.185859 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.185837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.186085 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.185873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.186085 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.185902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whg8x\" (UniqueName: \"kubernetes.io/projected/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-kube-api-access-whg8x\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.186085 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.185946    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.186245 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.186121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q2hm\" (UniqueName: \"kubernetes.io/projected/cfe74f93-4384-4f79-8b94-01b01927224a-kube-api-access-8q2hm\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.186245 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.186214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.186330 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.186252    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.194896 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.194874    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whg8x\" (UniqueName: \"kubernetes.io/projected/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-kube-api-access-whg8x\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.228925 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.228903    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"]
Aug 02 10:44:27.232045 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.232015    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.243063 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.243042    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"]
Aug 02 10:44:27.252242 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.252224    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:27.286804 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.286776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.286931 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.286817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.286931 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.286844    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.286931 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.286877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q2hm\" (UniqueName: \"kubernetes.io/projected/cfe74f93-4384-4f79-8b94-01b01927224a-kube-api-access-8q2hm\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.286931 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.286909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.287149 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.287074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gm64\" (UniqueName: \"kubernetes.io/projected/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-kube-api-access-2gm64\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.287272 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.287258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.287304 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.287254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.296304 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.296280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q2hm\" (UniqueName: \"kubernetes.io/projected/cfe74f93-4384-4f79-8b94-01b01927224a-kube-api-access-8q2hm\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.337362 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.337337    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:27.385005 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.384984    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"]
Aug 02 10:44:27.387349 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:27.387298    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod127818aa_8adb_4c9a_92b1_daa1f1fc8f52.slice/crio-b4c44c24fc3ee689b6c29dea3c7dfaca301524bd62dd3535533bc3fd48a8cd3d WatchSource:0}: Error finding container b4c44c24fc3ee689b6c29dea3c7dfaca301524bd62dd3535533bc3fd48a8cd3d: Status 404 returned error can't find the container with id b4c44c24fc3ee689b6c29dea3c7dfaca301524bd62dd3535533bc3fd48a8cd3d
Aug 02 10:44:27.387974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.387552    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.387974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.387595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.387974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.387635    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.387974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.387659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2rwk\" (UniqueName: \"kubernetes.io/projected/ccd78cda-4a6d-4afa-8caa-2847d87634f6-kube-api-access-p2rwk\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.387974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.387751    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.387974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.387783    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gm64\" (UniqueName: \"kubernetes.io/projected/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-kube-api-access-2gm64\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.387974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.387905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.388432 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.388361    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.397770 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.397741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gm64\" (UniqueName: \"kubernetes.io/projected/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-kube-api-access-2gm64\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.436449 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.436069    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:27.479321 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.479290    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"]
Aug 02 10:44:27.482761 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:27.482701    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfe74f93_4384_4f79_8b94_01b01927224a.slice/crio-7666dfed9a1c093f8f4a0699c8510326726716a1775b10bbe91503090a1cc457 WatchSource:0}: Error finding container 7666dfed9a1c093f8f4a0699c8510326726716a1775b10bbe91503090a1cc457: Status 404 returned error can't find the container with id 7666dfed9a1c093f8f4a0699c8510326726716a1775b10bbe91503090a1cc457
Aug 02 10:44:27.488687 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.488665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.488786 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.488712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2rwk\" (UniqueName: \"kubernetes.io/projected/ccd78cda-4a6d-4afa-8caa-2847d87634f6-kube-api-access-p2rwk\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.488846 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.488783    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.489276 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.489249    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.489342 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.489270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.499517 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.499491    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2rwk\" (UniqueName: \"kubernetes.io/projected/ccd78cda-4a6d-4afa-8caa-2847d87634f6-kube-api-access-p2rwk\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.541329 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.541300    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:27.585753 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.585729    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"]
Aug 02 10:44:27.588449 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:27.588424    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea53e1cc_1fc5_4093_b29b_e42c8c48068a.slice/crio-80fe22c63401bbecb644553e4c59e4d1b5c0620cf0a35c6f3ba160e5a6c78a70 WatchSource:0}: Error finding container 80fe22c63401bbecb644553e4c59e4d1b5c0620cf0a35c6f3ba160e5a6c78a70: Status 404 returned error can't find the container with id 80fe22c63401bbecb644553e4c59e4d1b5c0620cf0a35c6f3ba160e5a6c78a70
Aug 02 10:44:27.693753 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:27.693731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"]
Aug 02 10:44:27.696737 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:27.696701    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd78cda_4a6d_4afa_8caa_2847d87634f6.slice/crio-6f4593057891d08689af49768bb27f12e3bc56bf62daa0541d143400c1c2b59c WatchSource:0}: Error finding container 6f4593057891d08689af49768bb27f12e3bc56bf62daa0541d143400c1c2b59c: Status 404 returned error can't find the container with id 6f4593057891d08689af49768bb27f12e3bc56bf62daa0541d143400c1c2b59c
Aug 02 10:44:28.388247 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.388215    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfe74f93-4384-4f79-8b94-01b01927224a" containerID="9d0abacd36b4b799bebaf3be660c7d19e9462293171c359ea9f0bbb72093dc9c" exitCode=0
Aug 02 10:44:28.388727 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.388313    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll" event={"ID":"cfe74f93-4384-4f79-8b94-01b01927224a","Type":"ContainerDied","Data":"9d0abacd36b4b799bebaf3be660c7d19e9462293171c359ea9f0bbb72093dc9c"}
Aug 02 10:44:28.388727 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.388365    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll" event={"ID":"cfe74f93-4384-4f79-8b94-01b01927224a","Type":"ContainerStarted","Data":"7666dfed9a1c093f8f4a0699c8510326726716a1775b10bbe91503090a1cc457"}
Aug 02 10:44:28.389772 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.389701    2576 generic.go:356] "Generic (PLEG): container finished" podID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerID="d0a928ade5c91ac71891384407b3a14f34feb5bb9c191787ddc53370399c695f" exitCode=0
Aug 02 10:44:28.389825 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.389780    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc" event={"ID":"ccd78cda-4a6d-4afa-8caa-2847d87634f6","Type":"ContainerDied","Data":"d0a928ade5c91ac71891384407b3a14f34feb5bb9c191787ddc53370399c695f"}
Aug 02 10:44:28.389870 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.389822    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc" event={"ID":"ccd78cda-4a6d-4afa-8caa-2847d87634f6","Type":"ContainerStarted","Data":"6f4593057891d08689af49768bb27f12e3bc56bf62daa0541d143400c1c2b59c"}
Aug 02 10:44:28.391252 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.391230    2576 generic.go:356] "Generic (PLEG): container finished" podID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerID="bd1a9f63117cc9feb4f1887c122396e815a6a0684a56cd2d5d60dea0739b9480" exitCode=0
Aug 02 10:44:28.391365 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.391255    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65" event={"ID":"127818aa-8adb-4c9a-92b1-daa1f1fc8f52","Type":"ContainerDied","Data":"bd1a9f63117cc9feb4f1887c122396e815a6a0684a56cd2d5d60dea0739b9480"}
Aug 02 10:44:28.391365 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.391290    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65" event={"ID":"127818aa-8adb-4c9a-92b1-daa1f1fc8f52","Type":"ContainerStarted","Data":"b4c44c24fc3ee689b6c29dea3c7dfaca301524bd62dd3535533bc3fd48a8cd3d"}
Aug 02 10:44:28.392774 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.392755    2576 generic.go:356] "Generic (PLEG): container finished" podID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerID="1db45020815a167a42934ea271d854ba3fa68b22060f51be52a783f80ac32c97" exitCode=0
Aug 02 10:44:28.392838 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.392806    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx" event={"ID":"ea53e1cc-1fc5-4093-b29b-e42c8c48068a","Type":"ContainerDied","Data":"1db45020815a167a42934ea271d854ba3fa68b22060f51be52a783f80ac32c97"}
Aug 02 10:44:28.392885 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:28.392836    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx" event={"ID":"ea53e1cc-1fc5-4093-b29b-e42c8c48068a","Type":"ContainerStarted","Data":"80fe22c63401bbecb644553e4c59e4d1b5c0620cf0a35c6f3ba160e5a6c78a70"}
Aug 02 10:44:29.401312 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:29.401273    2576 generic.go:356] "Generic (PLEG): container finished" podID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerID="6d2b8ccdc2eec8a49a6ba745a06b36b772da35fef2c0ac6f9468acb805000cd3" exitCode=0
Aug 02 10:44:29.401779 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:29.401314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65" event={"ID":"127818aa-8adb-4c9a-92b1-daa1f1fc8f52","Type":"ContainerDied","Data":"6d2b8ccdc2eec8a49a6ba745a06b36b772da35fef2c0ac6f9468acb805000cd3"}
Aug 02 10:44:30.409592 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:30.409555    2576 generic.go:356] "Generic (PLEG): container finished" podID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerID="cdf9b59a016e411b7a4a25aef2ded508772c267daa14ae3c2bb83800c5031111" exitCode=0
Aug 02 10:44:30.410054 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:30.409645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc" event={"ID":"ccd78cda-4a6d-4afa-8caa-2847d87634f6","Type":"ContainerDied","Data":"cdf9b59a016e411b7a4a25aef2ded508772c267daa14ae3c2bb83800c5031111"}
Aug 02 10:44:30.411865 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:30.411841    2576 generic.go:356] "Generic (PLEG): container finished" podID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerID="bae7bfa09b9962ea0ccf5d3531c62087e9d74ae3ac357e606262122e5cf0890e" exitCode=0
Aug 02 10:44:30.411947 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:30.411917    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65" event={"ID":"127818aa-8adb-4c9a-92b1-daa1f1fc8f52","Type":"ContainerDied","Data":"bae7bfa09b9962ea0ccf5d3531c62087e9d74ae3ac357e606262122e5cf0890e"}
Aug 02 10:44:30.413641 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:30.413621    2576 generic.go:356] "Generic (PLEG): container finished" podID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerID="3e661ca2a19961e7a13514ea37165c47a6698cc5440c37fed33e180ca5bdf3bf" exitCode=0
Aug 02 10:44:30.413736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:30.413702    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx" event={"ID":"ea53e1cc-1fc5-4093-b29b-e42c8c48068a","Type":"ContainerDied","Data":"3e661ca2a19961e7a13514ea37165c47a6698cc5440c37fed33e180ca5bdf3bf"}
Aug 02 10:44:30.415686 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:30.415667    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfe74f93-4384-4f79-8b94-01b01927224a" containerID="5c368caa5bf7ec084e1446fae50a22aa7e71ed2b046bb8a992c70fe863131f7c" exitCode=0
Aug 02 10:44:30.415766 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:30.415720    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll" event={"ID":"cfe74f93-4384-4f79-8b94-01b01927224a","Type":"ContainerDied","Data":"5c368caa5bf7ec084e1446fae50a22aa7e71ed2b046bb8a992c70fe863131f7c"}
Aug 02 10:44:31.421918 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.421884    2576 generic.go:356] "Generic (PLEG): container finished" podID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerID="514dac1999d25080500eb446150a1259ef325fd2135521203dad0757c6826067" exitCode=0
Aug 02 10:44:31.422316 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.421975    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx" event={"ID":"ea53e1cc-1fc5-4093-b29b-e42c8c48068a","Type":"ContainerDied","Data":"514dac1999d25080500eb446150a1259ef325fd2135521203dad0757c6826067"}
Aug 02 10:44:31.423875 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.423850    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfe74f93-4384-4f79-8b94-01b01927224a" containerID="1137e428c3c3a84282f162ab6c8631b727eb2d55ef6741c89392dacacad4e339" exitCode=0
Aug 02 10:44:31.423983 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.423920    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll" event={"ID":"cfe74f93-4384-4f79-8b94-01b01927224a","Type":"ContainerDied","Data":"1137e428c3c3a84282f162ab6c8631b727eb2d55ef6741c89392dacacad4e339"}
Aug 02 10:44:31.425897 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.425875    2576 generic.go:356] "Generic (PLEG): container finished" podID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerID="aacffe982fd599d3a7f7ce9e981661aa5dc4d92c58d25b5f808a895e14ecc587" exitCode=0
Aug 02 10:44:31.425994 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.425962    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc" event={"ID":"ccd78cda-4a6d-4afa-8caa-2847d87634f6","Type":"ContainerDied","Data":"aacffe982fd599d3a7f7ce9e981661aa5dc4d92c58d25b5f808a895e14ecc587"}
Aug 02 10:44:31.562508 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.562483    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:31.728491 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.728457    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-bundle\") pod \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") "
Aug 02 10:44:31.728668 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.728512    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-util\") pod \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") "
Aug 02 10:44:31.728668 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.728537    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whg8x\" (UniqueName: \"kubernetes.io/projected/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-kube-api-access-whg8x\") pod \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\" (UID: \"127818aa-8adb-4c9a-92b1-daa1f1fc8f52\") "
Aug 02 10:44:31.728932 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.728901    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-bundle" (OuterVolumeSpecName: "bundle") pod "127818aa-8adb-4c9a-92b1-daa1f1fc8f52" (UID: "127818aa-8adb-4c9a-92b1-daa1f1fc8f52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:44:31.730714 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.730689    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-kube-api-access-whg8x" (OuterVolumeSpecName: "kube-api-access-whg8x") pod "127818aa-8adb-4c9a-92b1-daa1f1fc8f52" (UID: "127818aa-8adb-4c9a-92b1-daa1f1fc8f52"). InnerVolumeSpecName "kube-api-access-whg8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:44:31.734594 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.734559    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-util" (OuterVolumeSpecName: "util") pod "127818aa-8adb-4c9a-92b1-daa1f1fc8f52" (UID: "127818aa-8adb-4c9a-92b1-daa1f1fc8f52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:44:31.829694 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.829608    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:31.829694 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.829640    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-util\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:31.829694 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:31.829649    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whg8x\" (UniqueName: \"kubernetes.io/projected/127818aa-8adb-4c9a-92b1-daa1f1fc8f52-kube-api-access-whg8x\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:32.432702 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.432667    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65"
Aug 02 10:44:32.433214 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.432704    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0qfh65" event={"ID":"127818aa-8adb-4c9a-92b1-daa1f1fc8f52","Type":"ContainerDied","Data":"b4c44c24fc3ee689b6c29dea3c7dfaca301524bd62dd3535533bc3fd48a8cd3d"}
Aug 02 10:44:32.433214 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.432747    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4c44c24fc3ee689b6c29dea3c7dfaca301524bd62dd3535533bc3fd48a8cd3d"
Aug 02 10:44:32.585832 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.585804    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:32.618267 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.618246    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:32.621187 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.621171    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:32.737631 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.737549    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-bundle\") pod \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") "
Aug 02 10:44:32.737631 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.737588    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2rwk\" (UniqueName: \"kubernetes.io/projected/ccd78cda-4a6d-4afa-8caa-2847d87634f6-kube-api-access-p2rwk\") pod \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") "
Aug 02 10:44:32.737631 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.737608    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8q2hm\" (UniqueName: \"kubernetes.io/projected/cfe74f93-4384-4f79-8b94-01b01927224a-kube-api-access-8q2hm\") pod \"cfe74f93-4384-4f79-8b94-01b01927224a\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") "
Aug 02 10:44:32.737631 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.737627    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-bundle\") pod \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") "
Aug 02 10:44:32.737943 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.737657    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-util\") pod \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") "
Aug 02 10:44:32.737943 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.737704    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gm64\" (UniqueName: \"kubernetes.io/projected/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-kube-api-access-2gm64\") pod \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\" (UID: \"ea53e1cc-1fc5-4093-b29b-e42c8c48068a\") "
Aug 02 10:44:32.737943 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.737718    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-util\") pod \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\" (UID: \"ccd78cda-4a6d-4afa-8caa-2847d87634f6\") "
Aug 02 10:44:32.737943 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.737755    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-util\") pod \"cfe74f93-4384-4f79-8b94-01b01927224a\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") "
Aug 02 10:44:32.737943 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.737776    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-bundle\") pod \"cfe74f93-4384-4f79-8b94-01b01927224a\" (UID: \"cfe74f93-4384-4f79-8b94-01b01927224a\") "
Aug 02 10:44:32.738749 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.738586    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-bundle" (OuterVolumeSpecName: "bundle") pod "ea53e1cc-1fc5-4093-b29b-e42c8c48068a" (UID: "ea53e1cc-1fc5-4093-b29b-e42c8c48068a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:44:32.738749 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.738614    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-bundle" (OuterVolumeSpecName: "bundle") pod "ccd78cda-4a6d-4afa-8caa-2847d87634f6" (UID: "ccd78cda-4a6d-4afa-8caa-2847d87634f6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:44:32.738749 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.738683    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-bundle" (OuterVolumeSpecName: "bundle") pod "cfe74f93-4384-4f79-8b94-01b01927224a" (UID: "cfe74f93-4384-4f79-8b94-01b01927224a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:44:32.740406 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.740375    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe74f93-4384-4f79-8b94-01b01927224a-kube-api-access-8q2hm" (OuterVolumeSpecName: "kube-api-access-8q2hm") pod "cfe74f93-4384-4f79-8b94-01b01927224a" (UID: "cfe74f93-4384-4f79-8b94-01b01927224a"). InnerVolumeSpecName "kube-api-access-8q2hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:44:32.740596 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.740573    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccd78cda-4a6d-4afa-8caa-2847d87634f6-kube-api-access-p2rwk" (OuterVolumeSpecName: "kube-api-access-p2rwk") pod "ccd78cda-4a6d-4afa-8caa-2847d87634f6" (UID: "ccd78cda-4a6d-4afa-8caa-2847d87634f6"). InnerVolumeSpecName "kube-api-access-p2rwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:44:32.740871 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.740849    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-kube-api-access-2gm64" (OuterVolumeSpecName: "kube-api-access-2gm64") pod "ea53e1cc-1fc5-4093-b29b-e42c8c48068a" (UID: "ea53e1cc-1fc5-4093-b29b-e42c8c48068a"). InnerVolumeSpecName "kube-api-access-2gm64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:44:32.744308 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.744274    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-util" (OuterVolumeSpecName: "util") pod "ea53e1cc-1fc5-4093-b29b-e42c8c48068a" (UID: "ea53e1cc-1fc5-4093-b29b-e42c8c48068a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:44:32.745205 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.745186    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-util" (OuterVolumeSpecName: "util") pod "cfe74f93-4384-4f79-8b94-01b01927224a" (UID: "cfe74f93-4384-4f79-8b94-01b01927224a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:44:32.746778 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.746751    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-util" (OuterVolumeSpecName: "util") pod "ccd78cda-4a6d-4afa-8caa-2847d87634f6" (UID: "ccd78cda-4a6d-4afa-8caa-2847d87634f6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:44:32.838977 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.838942    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gm64\" (UniqueName: \"kubernetes.io/projected/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-kube-api-access-2gm64\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:32.838977 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.838969    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-util\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:32.838977 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.838980    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-util\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:32.838977 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.838987    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfe74f93-4384-4f79-8b94-01b01927224a-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:32.839256 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.838995    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ccd78cda-4a6d-4afa-8caa-2847d87634f6-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:32.839256 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.839003    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2rwk\" (UniqueName: \"kubernetes.io/projected/ccd78cda-4a6d-4afa-8caa-2847d87634f6-kube-api-access-p2rwk\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:32.839256 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.839011    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q2hm\" (UniqueName: \"kubernetes.io/projected/cfe74f93-4384-4f79-8b94-01b01927224a-kube-api-access-8q2hm\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:32.839256 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.839019    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:32.839256 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:32.839051    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea53e1cc-1fc5-4093-b29b-e42c8c48068a-util\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:33.439719 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:33.439689    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx"
Aug 02 10:44:33.439719 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:33.439696    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbw94nx" event={"ID":"ea53e1cc-1fc5-4093-b29b-e42c8c48068a","Type":"ContainerDied","Data":"80fe22c63401bbecb644553e4c59e4d1b5c0620cf0a35c6f3ba160e5a6c78a70"}
Aug 02 10:44:33.440212 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:33.439743    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80fe22c63401bbecb644553e4c59e4d1b5c0620cf0a35c6f3ba160e5a6c78a70"
Aug 02 10:44:33.441818 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:33.441794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll" event={"ID":"cfe74f93-4384-4f79-8b94-01b01927224a","Type":"ContainerDied","Data":"7666dfed9a1c093f8f4a0699c8510326726716a1775b10bbe91503090a1cc457"}
Aug 02 10:44:33.441971 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:33.441844    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7666dfed9a1c093f8f4a0699c8510326726716a1775b10bbe91503090a1cc457"
Aug 02 10:44:33.441971 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:33.441799    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705357fll"
Aug 02 10:44:33.444142 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:33.444126    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc"
Aug 02 10:44:33.444277 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:33.444125    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cq9jxc" event={"ID":"ccd78cda-4a6d-4afa-8caa-2847d87634f6","Type":"ContainerDied","Data":"6f4593057891d08689af49768bb27f12e3bc56bf62daa0541d143400c1c2b59c"}
Aug 02 10:44:33.444329 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:33.444280    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f4593057891d08689af49768bb27f12e3bc56bf62daa0541d143400c1c2b59c"
Aug 02 10:44:40.373498 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373465    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"]
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373767    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe74f93-4384-4f79-8b94-01b01927224a" containerName="util"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373777    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe74f93-4384-4f79-8b94-01b01927224a" containerName="util"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373786    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerName="util"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373791    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerName="util"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373800    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerName="extract"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373806    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerName="extract"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373813    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerName="pull"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373818    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerName="pull"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373825    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerName="util"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373829    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerName="util"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373834    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerName="pull"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373839    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerName="pull"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373850    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerName="extract"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373855    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerName="extract"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373860    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerName="util"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373865    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerName="util"
Aug 02 10:44:40.373862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373871    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe74f93-4384-4f79-8b94-01b01927224a" containerName="extract"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373876    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe74f93-4384-4f79-8b94-01b01927224a" containerName="extract"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373882    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe74f93-4384-4f79-8b94-01b01927224a" containerName="pull"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373887    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe74f93-4384-4f79-8b94-01b01927224a" containerName="pull"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373896    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerName="extract"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373900    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerName="extract"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373908    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerName="pull"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373913    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerName="pull"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373955    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe74f93-4384-4f79-8b94-01b01927224a" containerName="extract"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373962    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea53e1cc-1fc5-4093-b29b-e42c8c48068a" containerName="extract"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373969    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="127818aa-8adb-4c9a-92b1-daa1f1fc8f52" containerName="extract"
Aug 02 10:44:40.374414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.373976    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccd78cda-4a6d-4afa-8caa-2847d87634f6" containerName="extract"
Aug 02 10:44:40.379511 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.379481    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:40.383163 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.383140    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fjhc6\""
Aug 02 10:44:40.383305 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.383257    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 02 10:44:40.383477 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.383462    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 02 10:44:40.392099 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.392079    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"]
Aug 02 10:44:40.397699 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.397666    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7773114f-719c-4a0b-a2da-9d594a8f490f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" (UID: \"7773114f-719c-4a0b-a2da-9d594a8f490f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:40.397823 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.397710    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj8zf\" (UniqueName: \"kubernetes.io/projected/7773114f-719c-4a0b-a2da-9d594a8f490f-kube-api-access-kj8zf\") pod \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" (UID: \"7773114f-719c-4a0b-a2da-9d594a8f490f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:40.498381 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.498345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7773114f-719c-4a0b-a2da-9d594a8f490f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" (UID: \"7773114f-719c-4a0b-a2da-9d594a8f490f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:40.498522 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.498386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj8zf\" (UniqueName: \"kubernetes.io/projected/7773114f-719c-4a0b-a2da-9d594a8f490f-kube-api-access-kj8zf\") pod \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" (UID: \"7773114f-719c-4a0b-a2da-9d594a8f490f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:40.498705 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.498675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7773114f-719c-4a0b-a2da-9d594a8f490f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" (UID: \"7773114f-719c-4a0b-a2da-9d594a8f490f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:40.508524 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.508498    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj8zf\" (UniqueName: \"kubernetes.io/projected/7773114f-719c-4a0b-a2da-9d594a8f490f-kube-api-access-kj8zf\") pod \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" (UID: \"7773114f-719c-4a0b-a2da-9d594a8f490f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:40.689954 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.689925    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:40.827880 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:40.827856    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"]
Aug 02 10:44:40.829728 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:40.829702    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7773114f_719c_4a0b_a2da_9d594a8f490f.slice/crio-61fac372bd678c30fc0597bad172cb1a33df01569a1e525434871ec70d10f5af WatchSource:0}: Error finding container 61fac372bd678c30fc0597bad172cb1a33df01569a1e525434871ec70d10f5af: Status 404 returned error can't find the container with id 61fac372bd678c30fc0597bad172cb1a33df01569a1e525434871ec70d10f5af
Aug 02 10:44:41.484497 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:41.484462    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk" event={"ID":"7773114f-719c-4a0b-a2da-9d594a8f490f","Type":"ContainerStarted","Data":"61fac372bd678c30fc0597bad172cb1a33df01569a1e525434871ec70d10f5af"}
Aug 02 10:44:43.052596 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:43.051962    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-5t8k2"]
Aug 02 10:44:43.064228 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:43.064201    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2"
Aug 02 10:44:43.067344 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:43.067313    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-2pbpk\""
Aug 02 10:44:43.074978 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:43.074730    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-5t8k2"]
Aug 02 10:44:43.123677 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:43.123646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjt79\" (UniqueName: \"kubernetes.io/projected/4da1e27b-2768-4a33-a326-ddd5a661f677-kube-api-access-rjt79\") pod \"authorino-operator-68d88948d8-5t8k2\" (UID: \"4da1e27b-2768-4a33-a326-ddd5a661f677\") " pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2"
Aug 02 10:44:43.224558 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:43.224519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjt79\" (UniqueName: \"kubernetes.io/projected/4da1e27b-2768-4a33-a326-ddd5a661f677-kube-api-access-rjt79\") pod \"authorino-operator-68d88948d8-5t8k2\" (UID: \"4da1e27b-2768-4a33-a326-ddd5a661f677\") " pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2"
Aug 02 10:44:43.234730 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:43.234703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjt79\" (UniqueName: \"kubernetes.io/projected/4da1e27b-2768-4a33-a326-ddd5a661f677-kube-api-access-rjt79\") pod \"authorino-operator-68d88948d8-5t8k2\" (UID: \"4da1e27b-2768-4a33-a326-ddd5a661f677\") " pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2"
Aug 02 10:44:43.380779 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:43.380704    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2"
Aug 02 10:44:43.535818 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:43.535791    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-5t8k2"]
Aug 02 10:44:44.739358 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:44.739327    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4da1e27b_2768_4a33_a326_ddd5a661f677.slice/crio-5994e9d381b2510c5763d95af7bd0e5f7a2f5d9af998fc91f3a0ff727590f26b WatchSource:0}: Error finding container 5994e9d381b2510c5763d95af7bd0e5f7a2f5d9af998fc91f3a0ff727590f26b: Status 404 returned error can't find the container with id 5994e9d381b2510c5763d95af7bd0e5f7a2f5d9af998fc91f3a0ff727590f26b
Aug 02 10:44:45.240558 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.240525    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"]
Aug 02 10:44:45.243842 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.243823    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:45.247507 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.247487    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-szk9s\""
Aug 02 10:44:45.257957 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.257934    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"]
Aug 02 10:44:45.342961 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.342923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9c9g\" (UniqueName: \"kubernetes.io/projected/749c6ec6-f088-47f1-9af9-4a9ff219003a-kube-api-access-h9c9g\") pod \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" (UID: \"749c6ec6-f088-47f1-9af9-4a9ff219003a\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:45.443950 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.443916    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9c9g\" (UniqueName: \"kubernetes.io/projected/749c6ec6-f088-47f1-9af9-4a9ff219003a-kube-api-access-h9c9g\") pod \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" (UID: \"749c6ec6-f088-47f1-9af9-4a9ff219003a\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:45.454677 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.454646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9c9g\" (UniqueName: \"kubernetes.io/projected/749c6ec6-f088-47f1-9af9-4a9ff219003a-kube-api-access-h9c9g\") pod \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" (UID: \"749c6ec6-f088-47f1-9af9-4a9ff219003a\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:45.511370 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.511299    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk" event={"ID":"7773114f-719c-4a0b-a2da-9d594a8f490f","Type":"ContainerStarted","Data":"f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78"}
Aug 02 10:44:45.511483 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.511468    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:45.512635 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.512619    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2" event={"ID":"4da1e27b-2768-4a33-a326-ddd5a661f677","Type":"ContainerStarted","Data":"5994e9d381b2510c5763d95af7bd0e5f7a2f5d9af998fc91f3a0ff727590f26b"}
Aug 02 10:44:45.534359 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.534312    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk" podStartSLOduration=1.5557610099999999 podStartE2EDuration="5.534293482s" totalImagesPullingTime="3.978532472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:44:45.532470854 +0000 UTC m=+687.063053499" watchObservedRunningTime="2026-08-02 10:44:45.534293482 +0000 UTC m=+687.064876111"
Aug 02 10:44:45.553991 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.553969    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:45.688393 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:45.688365    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"]
Aug 02 10:44:45.690725 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:45.690693    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749c6ec6_f088_47f1_9af9_4a9ff219003a.slice/crio-36ea17f5da4f779921123e48ddda168c7fd750e6f77f2619c51b58a1d73deba1 WatchSource:0}: Error finding container 36ea17f5da4f779921123e48ddda168c7fd750e6f77f2619c51b58a1d73deba1: Status 404 returned error can't find the container with id 36ea17f5da4f779921123e48ddda168c7fd750e6f77f2619c51b58a1d73deba1
Aug 02 10:44:46.521630 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:46.521583    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8" event={"ID":"749c6ec6-f088-47f1-9af9-4a9ff219003a","Type":"ContainerStarted","Data":"36ea17f5da4f779921123e48ddda168c7fd750e6f77f2619c51b58a1d73deba1"}
Aug 02 10:44:47.528452 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.528421    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2" event={"ID":"4da1e27b-2768-4a33-a326-ddd5a661f677","Type":"ContainerStarted","Data":"5536fb9792cd9e406c5c24956987a216fd649a0a8873e84d5a422f3cf6934d28"}
Aug 02 10:44:47.528847 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.528624    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2"
Aug 02 10:44:47.530174 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.530150    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8" event={"ID":"749c6ec6-f088-47f1-9af9-4a9ff219003a","Type":"ContainerStarted","Data":"1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38"}
Aug 02 10:44:47.530281 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.530193    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:47.551419 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.551378    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2" podStartSLOduration=2.202784703 podStartE2EDuration="4.551367062s" totalImagesPullingTime="2.348582359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:44:47.550423603 +0000 UTC m=+689.081006231" watchObservedRunningTime="2026-08-02 10:44:47.551367062 +0000 UTC m=+689.081949690"
Aug 02 10:44:47.569464 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.569427    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8" podStartSLOduration=1.118306526 podStartE2EDuration="2.569413766s" totalImagesPullingTime="1.45110724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:44:47.569130854 +0000 UTC m=+689.099713482" watchObservedRunningTime="2026-08-02 10:44:47.569413766 +0000 UTC m=+689.099996395"
Aug 02 10:44:47.623195 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.623162    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"]
Aug 02 10:44:47.626516 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.626498    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"
Aug 02 10:44:47.630476 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.630407    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Aug 02 10:44:47.630476 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.630419    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7w2hz\""
Aug 02 10:44:47.639419 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.639395    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"]
Aug 02 10:44:47.664635 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.664608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg5rc\" (UniqueName: \"kubernetes.io/projected/6887945d-dd8d-4115-b9be-1b6c83c5a831-kube-api-access-vg5rc\") pod \"dns-operator-controller-manager-5985564776-w5dzt\" (UID: \"6887945d-dd8d-4115-b9be-1b6c83c5a831\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"
Aug 02 10:44:47.765963 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.765933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg5rc\" (UniqueName: \"kubernetes.io/projected/6887945d-dd8d-4115-b9be-1b6c83c5a831-kube-api-access-vg5rc\") pod \"dns-operator-controller-manager-5985564776-w5dzt\" (UID: \"6887945d-dd8d-4115-b9be-1b6c83c5a831\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"
Aug 02 10:44:47.775225 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.775199    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg5rc\" (UniqueName: \"kubernetes.io/projected/6887945d-dd8d-4115-b9be-1b6c83c5a831-kube-api-access-vg5rc\") pod \"dns-operator-controller-manager-5985564776-w5dzt\" (UID: \"6887945d-dd8d-4115-b9be-1b6c83c5a831\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"
Aug 02 10:44:47.819158 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.819093    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"]
Aug 02 10:44:47.822434 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.822418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:47.825161 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.825143    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Aug 02 10:44:47.825252 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.825210    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zz2mr\""
Aug 02 10:44:47.825441 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.825429    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Aug 02 10:44:47.834601 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.834577    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"]
Aug 02 10:44:47.866348 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.866321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b7c9051c-65eb-4d50-b977-844178c1ec5b-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-k9fpw\" (UID: \"b7c9051c-65eb-4d50-b977-844178c1ec5b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:47.866447 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.866356    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x4fl\" (UniqueName: \"kubernetes.io/projected/b7c9051c-65eb-4d50-b977-844178c1ec5b-kube-api-access-6x4fl\") pod \"kuadrant-console-plugin-598f86468b-k9fpw\" (UID: \"b7c9051c-65eb-4d50-b977-844178c1ec5b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:47.866447 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.866387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7c9051c-65eb-4d50-b977-844178c1ec5b-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-k9fpw\" (UID: \"b7c9051c-65eb-4d50-b977-844178c1ec5b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:47.936852 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.936828    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"
Aug 02 10:44:47.967902 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.967877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b7c9051c-65eb-4d50-b977-844178c1ec5b-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-k9fpw\" (UID: \"b7c9051c-65eb-4d50-b977-844178c1ec5b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:47.968016 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.967912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x4fl\" (UniqueName: \"kubernetes.io/projected/b7c9051c-65eb-4d50-b977-844178c1ec5b-kube-api-access-6x4fl\") pod \"kuadrant-console-plugin-598f86468b-k9fpw\" (UID: \"b7c9051c-65eb-4d50-b977-844178c1ec5b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:47.968016 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.967932    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7c9051c-65eb-4d50-b977-844178c1ec5b-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-k9fpw\" (UID: \"b7c9051c-65eb-4d50-b977-844178c1ec5b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:47.968719 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.968699    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b7c9051c-65eb-4d50-b977-844178c1ec5b-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-k9fpw\" (UID: \"b7c9051c-65eb-4d50-b977-844178c1ec5b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:47.970289 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.970272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7c9051c-65eb-4d50-b977-844178c1ec5b-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-k9fpw\" (UID: \"b7c9051c-65eb-4d50-b977-844178c1ec5b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:47.983807 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:47.983771    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x4fl\" (UniqueName: \"kubernetes.io/projected/b7c9051c-65eb-4d50-b977-844178c1ec5b-kube-api-access-6x4fl\") pod \"kuadrant-console-plugin-598f86468b-k9fpw\" (UID: \"b7c9051c-65eb-4d50-b977-844178c1ec5b\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:48.072688 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:48.072665    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"]
Aug 02 10:44:48.074262 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:48.074237    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6887945d_dd8d_4115_b9be_1b6c83c5a831.slice/crio-1cf0cbff4b0a1bed657bf9e8ff9e51336d8ab54bb527dcfeb0fe3d74ce027997 WatchSource:0}: Error finding container 1cf0cbff4b0a1bed657bf9e8ff9e51336d8ab54bb527dcfeb0fe3d74ce027997: Status 404 returned error can't find the container with id 1cf0cbff4b0a1bed657bf9e8ff9e51336d8ab54bb527dcfeb0fe3d74ce027997
Aug 02 10:44:48.133977 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:48.133951    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"
Aug 02 10:44:48.297505 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:48.297484    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw"]
Aug 02 10:44:48.298868 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:48.298847    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7c9051c_65eb_4d50_b977_844178c1ec5b.slice/crio-02f166eaf8b437fc8b51c1e52587d8a46c0c0c63873ed4eba11deb0291f92e79 WatchSource:0}: Error finding container 02f166eaf8b437fc8b51c1e52587d8a46c0c0c63873ed4eba11deb0291f92e79: Status 404 returned error can't find the container with id 02f166eaf8b437fc8b51c1e52587d8a46c0c0c63873ed4eba11deb0291f92e79
Aug 02 10:44:48.535821 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:48.535790    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw" event={"ID":"b7c9051c-65eb-4d50-b977-844178c1ec5b","Type":"ContainerStarted","Data":"02f166eaf8b437fc8b51c1e52587d8a46c0c0c63873ed4eba11deb0291f92e79"}
Aug 02 10:44:48.537175 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:48.537149    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt" event={"ID":"6887945d-dd8d-4115-b9be-1b6c83c5a831","Type":"ContainerStarted","Data":"1cf0cbff4b0a1bed657bf9e8ff9e51336d8ab54bb527dcfeb0fe3d74ce027997"}
Aug 02 10:44:50.553704 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:50.553663    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt" event={"ID":"6887945d-dd8d-4115-b9be-1b6c83c5a831","Type":"ContainerStarted","Data":"cce1c7f269800c3efa45b8e59af16d6260fc7a2ff45e7d4661644b9667cb5a3d"}
Aug 02 10:44:50.554196 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:50.553771    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"
Aug 02 10:44:50.572082 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:50.572009    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt" podStartSLOduration=1.640991274 podStartE2EDuration="3.571989382s" totalImagesPullingTime="1.930998108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:44:50.570337342 +0000 UTC m=+692.100919994" watchObservedRunningTime="2026-08-02 10:44:50.571989382 +0000 UTC m=+692.102572013"
Aug 02 10:44:53.571994 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:53.571959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw" event={"ID":"b7c9051c-65eb-4d50-b977-844178c1ec5b","Type":"ContainerStarted","Data":"fc38aed36b5ca009bad681e97e982dc09d37760c6de93fccf57d1160316ad379"}
Aug 02 10:44:53.592416 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:53.592373    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-k9fpw" podStartSLOduration=1.823841569 podStartE2EDuration="6.592358373s" totalImagesPullingTime="4.768516804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:44:53.590487771 +0000 UTC m=+695.121070462" watchObservedRunningTime="2026-08-02 10:44:53.592358373 +0000 UTC m=+695.122941002"
Aug 02 10:44:56.524474 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:56.524438    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.309385 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.309353    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"]
Aug 02 10:44:58.309757 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.309546    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" containerName="manager" containerID="cri-o://f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78" gracePeriod=2
Aug 02 10:44:58.326374 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.326349    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"]
Aug 02 10:44:58.333366 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.333325    2576 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7773114f-719c-4a0b-a2da-9d594a8f490f\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-pwfjk\": pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.333705 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.333680    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"]
Aug 02 10:44:58.334234 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.334214    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" containerName="manager"
Aug 02 10:44:58.334234 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.334237    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" containerName="manager"
Aug 02 10:44:58.334416 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.334346    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" containerName="manager"
Aug 02 10:44:58.337438 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.337415    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"]
Aug 02 10:44:58.337559 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.337538    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:44:58.337686 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.337605    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" containerName="manager" containerID="cri-o://1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38" gracePeriod=2
Aug 02 10:44:58.339760 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.339738    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.349982 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.349961    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"]
Aug 02 10:44:58.355348 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.355325    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"]
Aug 02 10:44:58.366261 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.366241    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"]
Aug 02 10:44:58.366869 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.366836    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" containerName="manager"
Aug 02 10:44:58.366869 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.366862    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" containerName="manager"
Aug 02 10:44:58.367043 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.366969    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" containerName="manager"
Aug 02 10:44:58.369196 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.369170    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.370278 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.370261    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"
Aug 02 10:44:58.371365 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.371337    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.384634 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.384612    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"]
Aug 02 10:44:58.398518 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.398493    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.401194 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.401167    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.453668 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.453636    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-mj8z7\" (UID: \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:44:58.453785 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.453759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v22px\" (UniqueName: \"kubernetes.io/projected/f6d40d28-5cb5-43cf-b045-eb1cae85ab8e-kube-api-access-v22px\") pod \"limitador-operator-controller-manager-6c549b6d85-z74s4\" (UID: \"f6d40d28-5cb5-43cf-b045-eb1cae85ab8e\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"
Aug 02 10:44:58.453853 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.453802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxs7h\" (UniqueName: \"kubernetes.io/projected/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-kube-api-access-dxs7h\") pod \"kuadrant-operator-controller-manager-58fc56757c-mj8z7\" (UID: \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:44:58.539224 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.539194    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-5t8k2"
Aug 02 10:44:58.542824 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.542793    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.554377 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.554353    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-mj8z7\" (UID: \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:44:58.554494 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.554409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v22px\" (UniqueName: \"kubernetes.io/projected/f6d40d28-5cb5-43cf-b045-eb1cae85ab8e-kube-api-access-v22px\") pod \"limitador-operator-controller-manager-6c549b6d85-z74s4\" (UID: \"f6d40d28-5cb5-43cf-b045-eb1cae85ab8e\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"
Aug 02 10:44:58.554494 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.554430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxs7h\" (UniqueName: \"kubernetes.io/projected/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-kube-api-access-dxs7h\") pod \"kuadrant-operator-controller-manager-58fc56757c-mj8z7\" (UID: \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:44:58.554792 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.554770    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-mj8z7\" (UID: \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:44:58.565831 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.565802    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.568365 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.568345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v22px\" (UniqueName: \"kubernetes.io/projected/f6d40d28-5cb5-43cf-b045-eb1cae85ab8e-kube-api-access-v22px\") pod \"limitador-operator-controller-manager-6c549b6d85-z74s4\" (UID: \"f6d40d28-5cb5-43cf-b045-eb1cae85ab8e\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"
Aug 02 10:44:58.571355 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.571332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxs7h\" (UniqueName: \"kubernetes.io/projected/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-kube-api-access-dxs7h\") pod \"kuadrant-operator-controller-manager-58fc56757c-mj8z7\" (UID: \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:44:58.590681 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.590659    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.592821 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.592802    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.593590 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.593568    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.597125 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.597092    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.598327 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.598307    2576 generic.go:356] "Generic (PLEG): container finished" podID="749c6ec6-f088-47f1-9af9-4a9ff219003a" containerID="1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38" exitCode=0
Aug 02 10:44:58.598418 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.598343    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.598459 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.598419    2576 scope.go:117] "RemoveContainer" containerID="1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38"
Aug 02 10:44:58.600076 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.600053    2576 generic.go:356] "Generic (PLEG): container finished" podID="7773114f-719c-4a0b-a2da-9d594a8f490f" containerID="f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78" exitCode=0
Aug 02 10:44:58.600177 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.600092    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.600776 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.600752    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.603191 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.603161    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.605715 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.605695    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.607889 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.607873    2576 scope.go:117] "RemoveContainer" containerID="1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38"
Aug 02 10:44:58.608074 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.608051    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.608215 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.608190    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38\": container with ID starting with 1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38 not found: ID does not exist" containerID="1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38"
Aug 02 10:44:58.608259 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.608227    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38"} err="failed to get container status \"1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38\": rpc error: code = NotFound desc = could not find container \"1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38\": container with ID starting with 1f45654b2cf28672e6eefee6c16f14c04259b2604a1d9ba42a61db555dc41d38 not found: ID does not exist"
Aug 02 10:44:58.608259 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.608250    2576 scope.go:117] "RemoveContainer" containerID="f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78"
Aug 02 10:44:58.615966 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.615951    2576 scope.go:117] "RemoveContainer" containerID="f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78"
Aug 02 10:44:58.616265 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.616241    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78\": container with ID starting with f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78 not found: ID does not exist" containerID="f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78"
Aug 02 10:44:58.616336 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.616275    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78"} err="failed to get container status \"f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78\": rpc error: code = NotFound desc = could not find container \"f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78\": container with ID starting with f1ff642838e0025e8e47e2c57b7a27650062d9eb51e0874cb2043b0fdcd3db78 not found: ID does not exist"
Aug 02 10:44:58.655306 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.655281    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9c9g\" (UniqueName: \"kubernetes.io/projected/749c6ec6-f088-47f1-9af9-4a9ff219003a-kube-api-access-h9c9g\") pod \"749c6ec6-f088-47f1-9af9-4a9ff219003a\" (UID: \"749c6ec6-f088-47f1-9af9-4a9ff219003a\") "
Aug 02 10:44:58.655415 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.655332    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7773114f-719c-4a0b-a2da-9d594a8f490f-extensions-socket-volume\") pod \"7773114f-719c-4a0b-a2da-9d594a8f490f\" (UID: \"7773114f-719c-4a0b-a2da-9d594a8f490f\") "
Aug 02 10:44:58.655415 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.655377    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj8zf\" (UniqueName: \"kubernetes.io/projected/7773114f-719c-4a0b-a2da-9d594a8f490f-kube-api-access-kj8zf\") pod \"7773114f-719c-4a0b-a2da-9d594a8f490f\" (UID: \"7773114f-719c-4a0b-a2da-9d594a8f490f\") "
Aug 02 10:44:58.655804 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.655771    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7773114f-719c-4a0b-a2da-9d594a8f490f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7773114f-719c-4a0b-a2da-9d594a8f490f" (UID: "7773114f-719c-4a0b-a2da-9d594a8f490f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:44:58.657583 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.657561    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7773114f-719c-4a0b-a2da-9d594a8f490f-kube-api-access-kj8zf" (OuterVolumeSpecName: "kube-api-access-kj8zf") pod "7773114f-719c-4a0b-a2da-9d594a8f490f" (UID: "7773114f-719c-4a0b-a2da-9d594a8f490f"). InnerVolumeSpecName "kube-api-access-kj8zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:44:58.657651 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.657612    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749c6ec6-f088-47f1-9af9-4a9ff219003a-kube-api-access-h9c9g" (OuterVolumeSpecName: "kube-api-access-h9c9g") pod "749c6ec6-f088-47f1-9af9-4a9ff219003a" (UID: "749c6ec6-f088-47f1-9af9-4a9ff219003a"). InnerVolumeSpecName "kube-api-access-h9c9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:44:58.755969 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.755944    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:44:58.756160 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.756145    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9c9g\" (UniqueName: \"kubernetes.io/projected/749c6ec6-f088-47f1-9af9-4a9ff219003a-kube-api-access-h9c9g\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:58.756200 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.756164    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7773114f-719c-4a0b-a2da-9d594a8f490f-extensions-socket-volume\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:58.756200 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.756173    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj8zf\" (UniqueName: \"kubernetes.io/projected/7773114f-719c-4a0b-a2da-9d594a8f490f-kube-api-access-kj8zf\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:44:58.763111 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.763094    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"
Aug 02 10:44:58.906411 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.906290    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"]
Aug 02 10:44:58.909252 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:58.909226    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625a4e0e_b63d_473b_9d9a_7c7d4352ab0b.slice/crio-ab471de75ef8690ca018e8f1e215ee0ee8606f96a36a93523563028512c9e6c4 WatchSource:0}: Error finding container ab471de75ef8690ca018e8f1e215ee0ee8606f96a36a93523563028512c9e6c4: Status 404 returned error can't find the container with id ab471de75ef8690ca018e8f1e215ee0ee8606f96a36a93523563028512c9e6c4
Aug 02 10:44:58.912716 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.912687    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.914725 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.914703    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.916867 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.916843    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.918900 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.918878    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:58.926497 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.926477    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"]
Aug 02 10:44:58.929281 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:44:58.929257    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d40d28_5cb5_43cf_b045_eb1cae85ab8e.slice/crio-a239d5baa87534a8549d97e0c9a2a3f4d0e3277bd4005528dc95fbd470337b92 WatchSource:0}: Error finding container a239d5baa87534a8549d97e0c9a2a3f4d0e3277bd4005528dc95fbd470337b92: Status 404 returned error can't find the container with id a239d5baa87534a8549d97e0c9a2a3f4d0e3277bd4005528dc95fbd470337b92
Aug 02 10:44:58.951236 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.951208    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-pwfjk\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-pwfjk"
Aug 02 10:44:58.952553 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.952528    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" path="/var/lib/kubelet/pods/749c6ec6-f088-47f1-9af9-4a9ff219003a/volumes"
Aug 02 10:44:58.952890 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:58.952872    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7773114f-719c-4a0b-a2da-9d594a8f490f" path="/var/lib/kubelet/pods/7773114f-719c-4a0b-a2da-9d594a8f490f/volumes"
Aug 02 10:44:58.953478 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:44:58.953454    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-bgdg8\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="749c6ec6-f088-47f1-9af9-4a9ff219003a" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-bgdg8"
Aug 02 10:44:59.611823 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.611740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7" event={"ID":"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b","Type":"ContainerStarted","Data":"88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0"}
Aug 02 10:44:59.611823 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.611803    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:44:59.612296 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.611827    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7" event={"ID":"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b","Type":"ContainerStarted","Data":"ab471de75ef8690ca018e8f1e215ee0ee8606f96a36a93523563028512c9e6c4"}
Aug 02 10:44:59.614192 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.614170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4" event={"ID":"f6d40d28-5cb5-43cf-b045-eb1cae85ab8e","Type":"ContainerStarted","Data":"5bb21c0daeea3f4660aafed77fd137de6222eada1985427a2f899e8d0a30b2a0"}
Aug 02 10:44:59.614295 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.614216    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4" event={"ID":"f6d40d28-5cb5-43cf-b045-eb1cae85ab8e","Type":"ContainerStarted","Data":"a239d5baa87534a8549d97e0c9a2a3f4d0e3277bd4005528dc95fbd470337b92"}
Aug 02 10:44:59.614295 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.614240    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"
Aug 02 10:44:59.643636 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.643592    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7" podStartSLOduration=1.643579025 podStartE2EDuration="1.643579025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:44:59.641685969 +0000 UTC m=+701.172268608" watchObservedRunningTime="2026-08-02 10:44:59.643579025 +0000 UTC m=+701.174161654"
Aug 02 10:44:59.663749 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.663707    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4" podStartSLOduration=1.6636950050000001 podStartE2EDuration="1.663695005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:44:59.661456243 +0000 UTC m=+701.192038871" watchObservedRunningTime="2026-08-02 10:44:59.663695005 +0000 UTC m=+701.194277635"
Aug 02 10:44:59.758461 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.758430    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"]
Aug 02 10:44:59.761842 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.761827    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:44:59.774935 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.774911    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"]
Aug 02 10:44:59.865919 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.865833    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae154b-f9ba-432b-94eb-892de780dcf9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-w9z8z\" (UID: \"7cae154b-f9ba-432b-94eb-892de780dcf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:44:59.865919 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.865899    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq78c\" (UniqueName: \"kubernetes.io/projected/7cae154b-f9ba-432b-94eb-892de780dcf9-kube-api-access-kq78c\") pod \"kuadrant-operator-controller-manager-68cf589589-w9z8z\" (UID: \"7cae154b-f9ba-432b-94eb-892de780dcf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:44:59.967239 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.967199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae154b-f9ba-432b-94eb-892de780dcf9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-w9z8z\" (UID: \"7cae154b-f9ba-432b-94eb-892de780dcf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:44:59.967400 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.967261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq78c\" (UniqueName: \"kubernetes.io/projected/7cae154b-f9ba-432b-94eb-892de780dcf9-kube-api-access-kq78c\") pod \"kuadrant-operator-controller-manager-68cf589589-w9z8z\" (UID: \"7cae154b-f9ba-432b-94eb-892de780dcf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:44:59.967885 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.967859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae154b-f9ba-432b-94eb-892de780dcf9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-w9z8z\" (UID: \"7cae154b-f9ba-432b-94eb-892de780dcf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:44:59.980084 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:44:59.980061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq78c\" (UniqueName: \"kubernetes.io/projected/7cae154b-f9ba-432b-94eb-892de780dcf9-kube-api-access-kq78c\") pod \"kuadrant-operator-controller-manager-68cf589589-w9z8z\" (UID: \"7cae154b-f9ba-432b-94eb-892de780dcf9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:45:00.071761 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:00.071735    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:45:00.211069 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:00.210955    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"]
Aug 02 10:45:00.213801 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:45:00.213775    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cae154b_f9ba_432b_94eb_892de780dcf9.slice/crio-d5d09d06f829d6c5361b7f90e28316420393bbb50f22e61dcd0df3242a5c2616 WatchSource:0}: Error finding container d5d09d06f829d6c5361b7f90e28316420393bbb50f22e61dcd0df3242a5c2616: Status 404 returned error can't find the container with id d5d09d06f829d6c5361b7f90e28316420393bbb50f22e61dcd0df3242a5c2616
Aug 02 10:45:00.621915 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:00.621870    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z" event={"ID":"7cae154b-f9ba-432b-94eb-892de780dcf9","Type":"ContainerStarted","Data":"385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9"}
Aug 02 10:45:00.622364 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:00.622203    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:45:00.622364 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:00.622237    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z" event={"ID":"7cae154b-f9ba-432b-94eb-892de780dcf9","Type":"ContainerStarted","Data":"d5d09d06f829d6c5361b7f90e28316420393bbb50f22e61dcd0df3242a5c2616"}
Aug 02 10:45:00.655501 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:00.655447    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z" podStartSLOduration=1.65543388 podStartE2EDuration="1.65543388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:44:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:45:00.653960313 +0000 UTC m=+702.184542942" watchObservedRunningTime="2026-08-02 10:45:00.65543388 +0000 UTC m=+702.186016508"
Aug 02 10:45:01.563140 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:01.563108    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w5dzt"
Aug 02 10:45:10.624450 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:10.624380    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:45:10.624450 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:10.624445    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-z74s4"
Aug 02 10:45:11.629636 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:11.629600    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:45:11.692717 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:11.692683    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"]
Aug 02 10:45:11.692915 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:11.692890    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7" podUID="625a4e0e-b63d-473b-9d9a-7c7d4352ab0b" containerName="manager" containerID="cri-o://88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0" gracePeriod=10
Aug 02 10:45:11.975668 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:11.975641    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:45:12.061255 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.061222    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxs7h\" (UniqueName: \"kubernetes.io/projected/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-kube-api-access-dxs7h\") pod \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\" (UID: \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\") "
Aug 02 10:45:12.061404 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.061292    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-extensions-socket-volume\") pod \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\" (UID: \"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b\") "
Aug 02 10:45:12.061664 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.061643    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "625a4e0e-b63d-473b-9d9a-7c7d4352ab0b" (UID: "625a4e0e-b63d-473b-9d9a-7c7d4352ab0b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:45:12.063452 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.063430    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-kube-api-access-dxs7h" (OuterVolumeSpecName: "kube-api-access-dxs7h") pod "625a4e0e-b63d-473b-9d9a-7c7d4352ab0b" (UID: "625a4e0e-b63d-473b-9d9a-7c7d4352ab0b"). InnerVolumeSpecName "kube-api-access-dxs7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:45:12.161938 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.161914    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxs7h\" (UniqueName: \"kubernetes.io/projected/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-kube-api-access-dxs7h\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:45:12.161938 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.161938    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b-extensions-socket-volume\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:45:12.692393 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.692359    2576 generic.go:356] "Generic (PLEG): container finished" podID="625a4e0e-b63d-473b-9d9a-7c7d4352ab0b" containerID="88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0" exitCode=0
Aug 02 10:45:12.692778 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.692401    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"
Aug 02 10:45:12.692778 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.692409    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7" event={"ID":"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b","Type":"ContainerDied","Data":"88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0"}
Aug 02 10:45:12.692778 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.692457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7" event={"ID":"625a4e0e-b63d-473b-9d9a-7c7d4352ab0b","Type":"ContainerDied","Data":"ab471de75ef8690ca018e8f1e215ee0ee8606f96a36a93523563028512c9e6c4"}
Aug 02 10:45:12.692778 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.692480    2576 scope.go:117] "RemoveContainer" containerID="88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0"
Aug 02 10:45:12.701843 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.701822    2576 scope.go:117] "RemoveContainer" containerID="88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0"
Aug 02 10:45:12.702163 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:45:12.702143    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0\": container with ID starting with 88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0 not found: ID does not exist" containerID="88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0"
Aug 02 10:45:12.702225 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.702172    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0"} err="failed to get container status \"88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0\": rpc error: code = NotFound desc = could not find container \"88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0\": container with ID starting with 88a5b87d088e7bdcfcf8d35736fdc0e4e5b10fdef79086b3ae9955cd1eaceed0 not found: ID does not exist"
Aug 02 10:45:12.715550 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.715524    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"]
Aug 02 10:45:12.722096 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.722076    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mj8z7"]
Aug 02 10:45:12.949713 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:12.949651    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="625a4e0e-b63d-473b-9d9a-7c7d4352ab0b" path="/var/lib/kubelet/pods/625a4e0e-b63d-473b-9d9a-7c7d4352ab0b/volumes"
Aug 02 10:45:27.986823 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:27.986786    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"]
Aug 02 10:45:27.987383 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:27.987331    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625a4e0e-b63d-473b-9d9a-7c7d4352ab0b" containerName="manager"
Aug 02 10:45:27.987383 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:27.987350    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="625a4e0e-b63d-473b-9d9a-7c7d4352ab0b" containerName="manager"
Aug 02 10:45:27.987497 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:27.987424    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="625a4e0e-b63d-473b-9d9a-7c7d4352ab0b" containerName="manager"
Aug 02 10:45:27.996239 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:27.996217    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:27.999010 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:27.998990    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-75xvs\""
Aug 02 10:45:27.999164 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:27.998990    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 02 10:45:28.004158 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.004130    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"]
Aug 02 10:45:28.104521 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.104481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.104720 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.104542    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzghz\" (UniqueName: \"kubernetes.io/projected/8b5b18a4-54de-4def-ae35-0cf0dab8deae-kube-api-access-qzghz\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.104720 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.104571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.104720 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.104629    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.104720 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.104681    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.104720 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.104706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.105002 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.104756    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.105002 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.104855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.105002 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.104894    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.206396 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.206396 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.206648 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.206648 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzghz\" (UniqueName: \"kubernetes.io/projected/8b5b18a4-54de-4def-ae35-0cf0dab8deae-kube-api-access-qzghz\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.206648 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.206648 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.206883 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.206883 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.206883 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206730    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.207072 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.206930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.207072 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.207057    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.207168 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.207084    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.207210 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.207180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.207398 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.207384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.209204 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.209168    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.209323 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.209278    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.214685 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.214662    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5b18a4-54de-4def-ae35-0cf0dab8deae-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.214817 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.214798    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzghz\" (UniqueName: \"kubernetes.io/projected/8b5b18a4-54de-4def-ae35-0cf0dab8deae-kube-api-access-qzghz\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dnqf6\" (UID: \"8b5b18a4-54de-4def-ae35-0cf0dab8deae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.308265 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.308178    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:28.449825 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.449796    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"]
Aug 02 10:45:28.451866 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:45:28.451840    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b5b18a4_54de_4def_ae35_0cf0dab8deae.slice/crio-98eced1e08cf2d164dd37a52481bfeebc51e59a526f1dc5826ae49150de98414 WatchSource:0}: Error finding container 98eced1e08cf2d164dd37a52481bfeebc51e59a526f1dc5826ae49150de98414: Status 404 returned error can't find the container with id 98eced1e08cf2d164dd37a52481bfeebc51e59a526f1dc5826ae49150de98414
Aug 02 10:45:28.776379 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:28.776341    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" event={"ID":"8b5b18a4-54de-4def-ae35-0cf0dab8deae","Type":"ContainerStarted","Data":"98eced1e08cf2d164dd37a52481bfeebc51e59a526f1dc5826ae49150de98414"}
Aug 02 10:45:31.028881 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:31.028846    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 10:45:31.029181 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:31.028928    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 10:45:31.029181 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:31.028968    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 10:45:31.795490 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:31.795451    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" event={"ID":"8b5b18a4-54de-4def-ae35-0cf0dab8deae","Type":"ContainerStarted","Data":"695f624951a1005e96565aa5725a81405ab538bc3858e0074639f83378b7edae"}
Aug 02 10:45:31.820643 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:31.820597    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" podStartSLOduration=2.245892103 podStartE2EDuration="4.820582003s" totalImagesPullingTime="2.5746899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:45:31.818550382 +0000 UTC m=+733.349133040" watchObservedRunningTime="2026-08-02 10:45:31.820582003 +0000 UTC m=+733.351164631"
Aug 02 10:45:32.308308 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.308271    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:32.313067 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.313021    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:32.802279 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.802247    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:32.803349 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.803321    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:45:32.804133 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.804113    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"]
Aug 02 10:45:32.806402 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.806386    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:32.810104 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.810082    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 02 10:45:32.816011 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.815991    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"]
Aug 02 10:45:32.896491 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.896461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"]
Aug 02 10:45:32.949603 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.949577    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bff975a-4a7a-468a-827f-847938720a3f-config-file\") pod \"limitador-limitador-859c4d96d6-h7jqj\" (UID: \"9bff975a-4a7a-468a-827f-847938720a3f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:32.949781 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:32.949728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klw2f\" (UniqueName: \"kubernetes.io/projected/9bff975a-4a7a-468a-827f-847938720a3f-kube-api-access-klw2f\") pod \"limitador-limitador-859c4d96d6-h7jqj\" (UID: \"9bff975a-4a7a-468a-827f-847938720a3f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:33.051294 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.051257    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klw2f\" (UniqueName: \"kubernetes.io/projected/9bff975a-4a7a-468a-827f-847938720a3f-kube-api-access-klw2f\") pod \"limitador-limitador-859c4d96d6-h7jqj\" (UID: \"9bff975a-4a7a-468a-827f-847938720a3f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:33.051533 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.051432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bff975a-4a7a-468a-827f-847938720a3f-config-file\") pod \"limitador-limitador-859c4d96d6-h7jqj\" (UID: \"9bff975a-4a7a-468a-827f-847938720a3f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:33.052067 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.052051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bff975a-4a7a-468a-827f-847938720a3f-config-file\") pod \"limitador-limitador-859c4d96d6-h7jqj\" (UID: \"9bff975a-4a7a-468a-827f-847938720a3f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:33.060080 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.060009    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klw2f\" (UniqueName: \"kubernetes.io/projected/9bff975a-4a7a-468a-827f-847938720a3f-kube-api-access-klw2f\") pod \"limitador-limitador-859c4d96d6-h7jqj\" (UID: \"9bff975a-4a7a-468a-827f-847938720a3f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:33.117003 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.116975    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:33.261903 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.261877    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"]
Aug 02 10:45:33.263572 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:45:33.263546    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bff975a_4a7a_468a_827f_847938720a3f.slice/crio-5da6b3eaf8cb182e9f8787036193e2ba87d3a01aa2bec145bd1b5ceac9a14c79 WatchSource:0}: Error finding container 5da6b3eaf8cb182e9f8787036193e2ba87d3a01aa2bec145bd1b5ceac9a14c79: Status 404 returned error can't find the container with id 5da6b3eaf8cb182e9f8787036193e2ba87d3a01aa2bec145bd1b5ceac9a14c79
Aug 02 10:45:33.807939 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.807910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj" event={"ID":"9bff975a-4a7a-468a-827f-847938720a3f","Type":"ContainerStarted","Data":"5da6b3eaf8cb182e9f8787036193e2ba87d3a01aa2bec145bd1b5ceac9a14c79"}
Aug 02 10:45:33.870319 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.870294    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-grth4"]
Aug 02 10:45:33.874936 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.874918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-grth4"
Aug 02 10:45:33.877726 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.877690    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jq9l6\""
Aug 02 10:45:33.880813 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:33.880794    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-grth4"]
Aug 02 10:45:34.061437 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:34.061345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57jpg\" (UniqueName: \"kubernetes.io/projected/2bfe0f05-c6bb-4afd-b132-6792b06cb721-kube-api-access-57jpg\") pod \"authorino-c548b5d7d-grth4\" (UID: \"2bfe0f05-c6bb-4afd-b132-6792b06cb721\") " pod="kuadrant-system/authorino-c548b5d7d-grth4"
Aug 02 10:45:34.162849 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:34.162807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57jpg\" (UniqueName: \"kubernetes.io/projected/2bfe0f05-c6bb-4afd-b132-6792b06cb721-kube-api-access-57jpg\") pod \"authorino-c548b5d7d-grth4\" (UID: \"2bfe0f05-c6bb-4afd-b132-6792b06cb721\") " pod="kuadrant-system/authorino-c548b5d7d-grth4"
Aug 02 10:45:34.171750 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:34.171728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57jpg\" (UniqueName: \"kubernetes.io/projected/2bfe0f05-c6bb-4afd-b132-6792b06cb721-kube-api-access-57jpg\") pod \"authorino-c548b5d7d-grth4\" (UID: \"2bfe0f05-c6bb-4afd-b132-6792b06cb721\") " pod="kuadrant-system/authorino-c548b5d7d-grth4"
Aug 02 10:45:34.185541 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:34.185519    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-grth4"
Aug 02 10:45:34.507276 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:34.507203    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-grth4"]
Aug 02 10:45:34.509855 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:45:34.509831    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bfe0f05_c6bb_4afd_b132_6792b06cb721.slice/crio-02946ad58e848c552aad16ecb2dc052a90a38a1ee6f485465a7323c63c1bd459 WatchSource:0}: Error finding container 02946ad58e848c552aad16ecb2dc052a90a38a1ee6f485465a7323c63c1bd459: Status 404 returned error can't find the container with id 02946ad58e848c552aad16ecb2dc052a90a38a1ee6f485465a7323c63c1bd459
Aug 02 10:45:34.820683 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:34.820645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj" event={"ID":"9bff975a-4a7a-468a-827f-847938720a3f","Type":"ContainerStarted","Data":"cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0"}
Aug 02 10:45:34.821168 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:34.820763    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:34.822144 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:34.822124    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-grth4" event={"ID":"2bfe0f05-c6bb-4afd-b132-6792b06cb721","Type":"ContainerStarted","Data":"02946ad58e848c552aad16ecb2dc052a90a38a1ee6f485465a7323c63c1bd459"}
Aug 02 10:45:34.838671 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:34.838630    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj" podStartSLOduration=1.660519077 podStartE2EDuration="2.838618292s" totalImagesPullingTime="1.178099215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:45:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:45:34.836873466 +0000 UTC m=+736.367456115" watchObservedRunningTime="2026-08-02 10:45:34.838618292 +0000 UTC m=+736.369200920"
Aug 02 10:45:36.834670 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:36.834635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-grth4" event={"ID":"2bfe0f05-c6bb-4afd-b132-6792b06cb721","Type":"ContainerStarted","Data":"4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426"}
Aug 02 10:45:36.851513 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:36.851470    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-grth4" podStartSLOduration=2.092708837 podStartE2EDuration="3.851455168s" totalImagesPullingTime="1.758746331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:45:36.849810059 +0000 UTC m=+738.380392688" watchObservedRunningTime="2026-08-02 10:45:36.851455168 +0000 UTC m=+738.382037978"
Aug 02 10:45:45.829222 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:45.829190    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:46.703600 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:46.703563    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"]
Aug 02 10:45:46.703883 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:46.703842    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj" podUID="9bff975a-4a7a-468a-827f-847938720a3f" containerName="limitador" containerID="cri-o://cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0" gracePeriod=30
Aug 02 10:45:47.651776 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.651752    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:47.675335 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.675304    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klw2f\" (UniqueName: \"kubernetes.io/projected/9bff975a-4a7a-468a-827f-847938720a3f-kube-api-access-klw2f\") pod \"9bff975a-4a7a-468a-827f-847938720a3f\" (UID: \"9bff975a-4a7a-468a-827f-847938720a3f\") "
Aug 02 10:45:47.675484 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.675370    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bff975a-4a7a-468a-827f-847938720a3f-config-file\") pod \"9bff975a-4a7a-468a-827f-847938720a3f\" (UID: \"9bff975a-4a7a-468a-827f-847938720a3f\") "
Aug 02 10:45:47.675756 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.675737    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bff975a-4a7a-468a-827f-847938720a3f-config-file" (OuterVolumeSpecName: "config-file") pod "9bff975a-4a7a-468a-827f-847938720a3f" (UID: "9bff975a-4a7a-468a-827f-847938720a3f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 10:45:47.677676 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.677647    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bff975a-4a7a-468a-827f-847938720a3f-kube-api-access-klw2f" (OuterVolumeSpecName: "kube-api-access-klw2f") pod "9bff975a-4a7a-468a-827f-847938720a3f" (UID: "9bff975a-4a7a-468a-827f-847938720a3f"). InnerVolumeSpecName "kube-api-access-klw2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:45:47.776245 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.776156    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klw2f\" (UniqueName: \"kubernetes.io/projected/9bff975a-4a7a-468a-827f-847938720a3f-kube-api-access-klw2f\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:45:47.776245 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.776187    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bff975a-4a7a-468a-827f-847938720a3f-config-file\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:45:47.897706 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.897674    2576 generic.go:356] "Generic (PLEG): container finished" podID="9bff975a-4a7a-468a-827f-847938720a3f" containerID="cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0" exitCode=0
Aug 02 10:45:47.897883 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.897733    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"
Aug 02 10:45:47.897883 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.897755    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj" event={"ID":"9bff975a-4a7a-468a-827f-847938720a3f","Type":"ContainerDied","Data":"cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0"}
Aug 02 10:45:47.897883 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.897824    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-h7jqj" event={"ID":"9bff975a-4a7a-468a-827f-847938720a3f","Type":"ContainerDied","Data":"5da6b3eaf8cb182e9f8787036193e2ba87d3a01aa2bec145bd1b5ceac9a14c79"}
Aug 02 10:45:47.897883 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.897849    2576 scope.go:117] "RemoveContainer" containerID="cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0"
Aug 02 10:45:47.906761 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.906734    2576 scope.go:117] "RemoveContainer" containerID="cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0"
Aug 02 10:45:47.907091 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:45:47.907061    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0\": container with ID starting with cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0 not found: ID does not exist" containerID="cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0"
Aug 02 10:45:47.907187 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.907100    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0"} err="failed to get container status \"cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0\": rpc error: code = NotFound desc = could not find container \"cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0\": container with ID starting with cdf041075d56aaa3448fb72b6c561f2ee3e72ce95dd1141167785941673e28a0 not found: ID does not exist"
Aug 02 10:45:47.922324 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.922301    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"]
Aug 02 10:45:47.926243 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:47.926222    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-h7jqj"]
Aug 02 10:45:48.952116 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:48.952080    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bff975a-4a7a-468a-827f-847938720a3f" path="/var/lib/kubelet/pods/9bff975a-4a7a-468a-827f-847938720a3f/volumes"
Aug 02 10:45:49.386605 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.386514    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"]
Aug 02 10:45:49.386905 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.386889    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bff975a-4a7a-468a-827f-847938720a3f" containerName="limitador"
Aug 02 10:45:49.386905 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.386905    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bff975a-4a7a-468a-827f-847938720a3f" containerName="limitador"
Aug 02 10:45:49.387056 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.386967    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bff975a-4a7a-468a-827f-847938720a3f" containerName="limitador"
Aug 02 10:45:49.391365 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.391346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:45:49.394011 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.393985    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 02 10:45:49.394160 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.394081    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Aug 02 10:45:49.394370 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.394356    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 02 10:45:49.395459 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.395434    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fj9gv\""
Aug 02 10:45:49.398699 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.398681    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"]
Aug 02 10:45:49.492265 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.492234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5c54d517-6a51-4953-abc7-23c5f605b83c-data\") pod \"postgres-69fb4594cf-kf5gx\" (UID: \"5c54d517-6a51-4953-abc7-23c5f605b83c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:45:49.492448 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.492278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbd6z\" (UniqueName: \"kubernetes.io/projected/5c54d517-6a51-4953-abc7-23c5f605b83c-kube-api-access-bbd6z\") pod \"postgres-69fb4594cf-kf5gx\" (UID: \"5c54d517-6a51-4953-abc7-23c5f605b83c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:45:49.593461 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.593425    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5c54d517-6a51-4953-abc7-23c5f605b83c-data\") pod \"postgres-69fb4594cf-kf5gx\" (UID: \"5c54d517-6a51-4953-abc7-23c5f605b83c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:45:49.593618 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.593469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbd6z\" (UniqueName: \"kubernetes.io/projected/5c54d517-6a51-4953-abc7-23c5f605b83c-kube-api-access-bbd6z\") pod \"postgres-69fb4594cf-kf5gx\" (UID: \"5c54d517-6a51-4953-abc7-23c5f605b83c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:45:49.593849 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.593829    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5c54d517-6a51-4953-abc7-23c5f605b83c-data\") pod \"postgres-69fb4594cf-kf5gx\" (UID: \"5c54d517-6a51-4953-abc7-23c5f605b83c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:45:49.602171 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.602150    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbd6z\" (UniqueName: \"kubernetes.io/projected/5c54d517-6a51-4953-abc7-23c5f605b83c-kube-api-access-bbd6z\") pod \"postgres-69fb4594cf-kf5gx\" (UID: \"5c54d517-6a51-4953-abc7-23c5f605b83c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:45:49.701894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.701871    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:45:49.837986 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.837962    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"]
Aug 02 10:45:49.840252 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:45:49.840228    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c54d517_6a51_4953_abc7_23c5f605b83c.slice/crio-3d79864c610c6351749ae26f4e61a497fe06b9b95b154e8448fbe6fc42aab734 WatchSource:0}: Error finding container 3d79864c610c6351749ae26f4e61a497fe06b9b95b154e8448fbe6fc42aab734: Status 404 returned error can't find the container with id 3d79864c610c6351749ae26f4e61a497fe06b9b95b154e8448fbe6fc42aab734
Aug 02 10:45:49.910656 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:49.910626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx" event={"ID":"5c54d517-6a51-4953-abc7-23c5f605b83c","Type":"ContainerStarted","Data":"3d79864c610c6351749ae26f4e61a497fe06b9b95b154e8448fbe6fc42aab734"}
Aug 02 10:45:54.949590 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:54.949557    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx" event={"ID":"5c54d517-6a51-4953-abc7-23c5f605b83c","Type":"ContainerStarted","Data":"084543e828e2ac1d728b182c7962702cae20bc2557e116e6fd97fdf87de4de66"}
Aug 02 10:45:54.949955 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:54.949610    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:45:54.964923 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:45:54.964833    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx" podStartSLOduration=1.092607751 podStartE2EDuration="5.964819241s" totalImagesPullingTime="4.87221149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:45:54.963597469 +0000 UTC m=+756.494180112" watchObservedRunningTime="2026-08-02 10:45:54.964819241 +0000 UTC m=+756.495401908"
Aug 02 10:46:00.979221 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:00.979190    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kf5gx"
Aug 02 10:46:01.775416 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:01.775385    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-597f7dd556-f68tj"]
Aug 02 10:46:01.781252 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:01.781231    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-597f7dd556-f68tj"
Aug 02 10:46:01.786122 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:01.786100    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-597f7dd556-f68tj"]
Aug 02 10:46:01.808205 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:01.808183    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-597f7dd556-f68tj"]
Aug 02 10:46:01.808472 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:46:01.808437    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-q2fv7], unattached volumes=[], failed to process volumes=[kube-api-access-q2fv7]: context canceled" pod="kuadrant-system/authorino-597f7dd556-f68tj" podUID="87b9ae20-b3e1-4342-aa5d-3647155176af"
Aug 02 10:46:01.899682 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:01.899645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2fv7\" (UniqueName: \"kubernetes.io/projected/87b9ae20-b3e1-4342-aa5d-3647155176af-kube-api-access-q2fv7\") pod \"authorino-597f7dd556-f68tj\" (UID: \"87b9ae20-b3e1-4342-aa5d-3647155176af\") " pod="kuadrant-system/authorino-597f7dd556-f68tj"
Aug 02 10:46:01.978688 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:01.978656    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-597f7dd556-f68tj"
Aug 02 10:46:01.985896 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:01.985874    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-597f7dd556-f68tj"
Aug 02 10:46:02.000343 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:02.000318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2fv7\" (UniqueName: \"kubernetes.io/projected/87b9ae20-b3e1-4342-aa5d-3647155176af-kube-api-access-q2fv7\") pod \"authorino-597f7dd556-f68tj\" (UID: \"87b9ae20-b3e1-4342-aa5d-3647155176af\") " pod="kuadrant-system/authorino-597f7dd556-f68tj"
Aug 02 10:46:02.009610 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:02.009584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2fv7\" (UniqueName: \"kubernetes.io/projected/87b9ae20-b3e1-4342-aa5d-3647155176af-kube-api-access-q2fv7\") pod \"authorino-597f7dd556-f68tj\" (UID: \"87b9ae20-b3e1-4342-aa5d-3647155176af\") " pod="kuadrant-system/authorino-597f7dd556-f68tj"
Aug 02 10:46:02.101552 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:02.101462    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2fv7\" (UniqueName: \"kubernetes.io/projected/87b9ae20-b3e1-4342-aa5d-3647155176af-kube-api-access-q2fv7\") pod \"87b9ae20-b3e1-4342-aa5d-3647155176af\" (UID: \"87b9ae20-b3e1-4342-aa5d-3647155176af\") "
Aug 02 10:46:02.103700 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:02.103673    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87b9ae20-b3e1-4342-aa5d-3647155176af-kube-api-access-q2fv7" (OuterVolumeSpecName: "kube-api-access-q2fv7") pod "87b9ae20-b3e1-4342-aa5d-3647155176af" (UID: "87b9ae20-b3e1-4342-aa5d-3647155176af"). InnerVolumeSpecName "kube-api-access-q2fv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:46:02.201997 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:02.201961    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2fv7\" (UniqueName: \"kubernetes.io/projected/87b9ae20-b3e1-4342-aa5d-3647155176af-kube-api-access-q2fv7\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:46:02.983151 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:02.983125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-597f7dd556-f68tj"
Aug 02 10:46:03.014506 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.014479    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-597f7dd556-f68tj"]
Aug 02 10:46:03.018885 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.018862    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-597f7dd556-f68tj"]
Aug 02 10:46:03.631121 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.631088    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-grth4"]
Aug 02 10:46:03.631277 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.631259    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-grth4" podUID="2bfe0f05-c6bb-4afd-b132-6792b06cb721" containerName="authorino" containerID="cri-o://4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426" gracePeriod=30
Aug 02 10:46:03.892232 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.892178    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-grth4"
Aug 02 10:46:03.989085 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.989050    2576 generic.go:356] "Generic (PLEG): container finished" podID="2bfe0f05-c6bb-4afd-b132-6792b06cb721" containerID="4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426" exitCode=0
Aug 02 10:46:03.989085 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.989083    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-grth4"
Aug 02 10:46:03.989293 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.989122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-grth4" event={"ID":"2bfe0f05-c6bb-4afd-b132-6792b06cb721","Type":"ContainerDied","Data":"4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426"}
Aug 02 10:46:03.989293 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.989167    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-grth4" event={"ID":"2bfe0f05-c6bb-4afd-b132-6792b06cb721","Type":"ContainerDied","Data":"02946ad58e848c552aad16ecb2dc052a90a38a1ee6f485465a7323c63c1bd459"}
Aug 02 10:46:03.989293 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.989185    2576 scope.go:117] "RemoveContainer" containerID="4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426"
Aug 02 10:46:03.998184 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.998166    2576 scope.go:117] "RemoveContainer" containerID="4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426"
Aug 02 10:46:03.998439 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:46:03.998423    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426\": container with ID starting with 4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426 not found: ID does not exist" containerID="4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426"
Aug 02 10:46:03.998484 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:03.998450    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426"} err="failed to get container status \"4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426\": rpc error: code = NotFound desc = could not find container \"4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426\": container with ID starting with 4fcd3e5a7dacd4dd4628ce35c8386a4fd609a26b79958a6e786e41869e9c9426 not found: ID does not exist"
Aug 02 10:46:04.017751 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:04.017717    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57jpg\" (UniqueName: \"kubernetes.io/projected/2bfe0f05-c6bb-4afd-b132-6792b06cb721-kube-api-access-57jpg\") pod \"2bfe0f05-c6bb-4afd-b132-6792b06cb721\" (UID: \"2bfe0f05-c6bb-4afd-b132-6792b06cb721\") "
Aug 02 10:46:04.019987 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:04.019961    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bfe0f05-c6bb-4afd-b132-6792b06cb721-kube-api-access-57jpg" (OuterVolumeSpecName: "kube-api-access-57jpg") pod "2bfe0f05-c6bb-4afd-b132-6792b06cb721" (UID: "2bfe0f05-c6bb-4afd-b132-6792b06cb721"). InnerVolumeSpecName "kube-api-access-57jpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:46:04.119045 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:04.118995    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57jpg\" (UniqueName: \"kubernetes.io/projected/2bfe0f05-c6bb-4afd-b132-6792b06cb721-kube-api-access-57jpg\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:46:04.314960 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:04.314931    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-grth4"]
Aug 02 10:46:04.323489 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:04.323467    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-grth4"]
Aug 02 10:46:04.950350 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:04.950320    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bfe0f05-c6bb-4afd-b132-6792b06cb721" path="/var/lib/kubelet/pods/2bfe0f05-c6bb-4afd-b132-6792b06cb721/volumes"
Aug 02 10:46:04.950642 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:04.950630    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87b9ae20-b3e1-4342-aa5d-3647155176af" path="/var/lib/kubelet/pods/87b9ae20-b3e1-4342-aa5d-3647155176af/volumes"
Aug 02 10:46:08.167284 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.167250    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-644f8d959b-5fkff"]
Aug 02 10:46:08.167718 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.167602    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bfe0f05-c6bb-4afd-b132-6792b06cb721" containerName="authorino"
Aug 02 10:46:08.167718 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.167612    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bfe0f05-c6bb-4afd-b132-6792b06cb721" containerName="authorino"
Aug 02 10:46:08.167718 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.167681    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bfe0f05-c6bb-4afd-b132-6792b06cb721" containerName="authorino"
Aug 02 10:46:08.291932 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.291892    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-644f8d959b-5fkff"]
Aug 02 10:46:08.292123 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.292015    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:08.295085 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.295067    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 02 10:46:08.295197 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.295084    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-76z9v\""
Aug 02 10:46:08.295197 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.295127    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 02 10:46:08.340881 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.340846    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57f59cbf46-2vhqv"]
Aug 02 10:46:08.343289 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.343271    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:08.355637 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.355616    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-2vhqv"]
Aug 02 10:46:08.455397 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.455309    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9233794d-93fe-40f2-bd83-c7ac8ef545df-webhook-cert\") pod \"maas-controller-644f8d959b-5fkff\" (UID: \"9233794d-93fe-40f2-bd83-c7ac8ef545df\") " pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:08.455397 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.455345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfzhn\" (UniqueName: \"kubernetes.io/projected/585a7939-76a1-428b-8546-5eca4a3ddeba-kube-api-access-nfzhn\") pod \"maas-controller-57f59cbf46-2vhqv\" (UID: \"585a7939-76a1-428b-8546-5eca4a3ddeba\") " pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:08.455611 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.455473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7496\" (UniqueName: \"kubernetes.io/projected/9233794d-93fe-40f2-bd83-c7ac8ef545df-kube-api-access-h7496\") pod \"maas-controller-644f8d959b-5fkff\" (UID: \"9233794d-93fe-40f2-bd83-c7ac8ef545df\") " pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:08.455611 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.455498    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/585a7939-76a1-428b-8546-5eca4a3ddeba-webhook-cert\") pod \"maas-controller-57f59cbf46-2vhqv\" (UID: \"585a7939-76a1-428b-8546-5eca4a3ddeba\") " pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:08.556441 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.556409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7496\" (UniqueName: \"kubernetes.io/projected/9233794d-93fe-40f2-bd83-c7ac8ef545df-kube-api-access-h7496\") pod \"maas-controller-644f8d959b-5fkff\" (UID: \"9233794d-93fe-40f2-bd83-c7ac8ef545df\") " pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:08.556441 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.556442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/585a7939-76a1-428b-8546-5eca4a3ddeba-webhook-cert\") pod \"maas-controller-57f59cbf46-2vhqv\" (UID: \"585a7939-76a1-428b-8546-5eca4a3ddeba\") " pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:08.556697 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.556462    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9233794d-93fe-40f2-bd83-c7ac8ef545df-webhook-cert\") pod \"maas-controller-644f8d959b-5fkff\" (UID: \"9233794d-93fe-40f2-bd83-c7ac8ef545df\") " pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:08.556697 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.556574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfzhn\" (UniqueName: \"kubernetes.io/projected/585a7939-76a1-428b-8546-5eca4a3ddeba-kube-api-access-nfzhn\") pod \"maas-controller-57f59cbf46-2vhqv\" (UID: \"585a7939-76a1-428b-8546-5eca4a3ddeba\") " pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:08.558954 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.558936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/585a7939-76a1-428b-8546-5eca4a3ddeba-webhook-cert\") pod \"maas-controller-57f59cbf46-2vhqv\" (UID: \"585a7939-76a1-428b-8546-5eca4a3ddeba\") " pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:08.559058 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.559020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9233794d-93fe-40f2-bd83-c7ac8ef545df-webhook-cert\") pod \"maas-controller-644f8d959b-5fkff\" (UID: \"9233794d-93fe-40f2-bd83-c7ac8ef545df\") " pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:08.565378 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.565353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfzhn\" (UniqueName: \"kubernetes.io/projected/585a7939-76a1-428b-8546-5eca4a3ddeba-kube-api-access-nfzhn\") pod \"maas-controller-57f59cbf46-2vhqv\" (UID: \"585a7939-76a1-428b-8546-5eca4a3ddeba\") " pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:08.565466 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.565425    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7496\" (UniqueName: \"kubernetes.io/projected/9233794d-93fe-40f2-bd83-c7ac8ef545df-kube-api-access-h7496\") pod \"maas-controller-644f8d959b-5fkff\" (UID: \"9233794d-93fe-40f2-bd83-c7ac8ef545df\") " pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:08.604414 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.604385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:08.656228 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.656200    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:08.748403 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.748372    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-644f8d959b-5fkff"]
Aug 02 10:46:08.749988 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:46:08.749963    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9233794d_93fe_40f2_bd83_c7ac8ef545df.slice/crio-3cd22ce63bef2efa4957378d4da12e83f79f2acfca764b4ca902d28ace936c80 WatchSource:0}: Error finding container 3cd22ce63bef2efa4957378d4da12e83f79f2acfca764b4ca902d28ace936c80: Status 404 returned error can't find the container with id 3cd22ce63bef2efa4957378d4da12e83f79f2acfca764b4ca902d28ace936c80
Aug 02 10:46:08.803512 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:08.803487    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-2vhqv"]
Aug 02 10:46:08.805168 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:46:08.805142    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585a7939_76a1_428b_8546_5eca4a3ddeba.slice/crio-e4d15f78534a14dd59ea9ff7f6f4238b3081778f85cf61f0e425404785d5ba94 WatchSource:0}: Error finding container e4d15f78534a14dd59ea9ff7f6f4238b3081778f85cf61f0e425404785d5ba94: Status 404 returned error can't find the container with id e4d15f78534a14dd59ea9ff7f6f4238b3081778f85cf61f0e425404785d5ba94
Aug 02 10:46:09.018110 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:09.017990    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-2vhqv" event={"ID":"585a7939-76a1-428b-8546-5eca4a3ddeba","Type":"ContainerStarted","Data":"e4d15f78534a14dd59ea9ff7f6f4238b3081778f85cf61f0e425404785d5ba94"}
Aug 02 10:46:09.019284 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:09.019257    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-644f8d959b-5fkff" event={"ID":"9233794d-93fe-40f2-bd83-c7ac8ef545df","Type":"ContainerStarted","Data":"3cd22ce63bef2efa4957378d4da12e83f79f2acfca764b4ca902d28ace936c80"}
Aug 02 10:46:11.031537 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:11.031487    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-2vhqv" event={"ID":"585a7939-76a1-428b-8546-5eca4a3ddeba","Type":"ContainerStarted","Data":"1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798"}
Aug 02 10:46:11.031968 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:11.031555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:11.033131 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:11.033103    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-644f8d959b-5fkff" event={"ID":"9233794d-93fe-40f2-bd83-c7ac8ef545df","Type":"ContainerStarted","Data":"ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2"}
Aug 02 10:46:11.033264 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:11.033151    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:11.060364 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:11.060317    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57f59cbf46-2vhqv" podStartSLOduration=1.7459017399999999 podStartE2EDuration="3.060300274s" totalImagesPullingTime="1.314398534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:46:11.058864707 +0000 UTC m=+772.589447356" watchObservedRunningTime="2026-08-02 10:46:11.060300274 +0000 UTC m=+772.590882903"
Aug 02 10:46:22.041209 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.041175    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:22.041643 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.041333    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:46:22.059564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.059517    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-644f8d959b-5fkff" podStartSLOduration=12.687745454 podStartE2EDuration="14.059501645s" totalImagesPullingTime="1.371756191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:46:11.110639251 +0000 UTC m=+772.641221910" watchObservedRunningTime="2026-08-02 10:46:22.059501645 +0000 UTC m=+783.590084273"
Aug 02 10:46:22.110478 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.110434    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-644f8d959b-5fkff"]
Aug 02 10:46:22.110737 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.110698    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-644f8d959b-5fkff" podUID="9233794d-93fe-40f2-bd83-c7ac8ef545df" containerName="manager" containerID="cri-o://ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2" gracePeriod=30
Aug 02 10:46:22.359554 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.359528    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:22.478198 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.478163    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7496\" (UniqueName: \"kubernetes.io/projected/9233794d-93fe-40f2-bd83-c7ac8ef545df-kube-api-access-h7496\") pod \"9233794d-93fe-40f2-bd83-c7ac8ef545df\" (UID: \"9233794d-93fe-40f2-bd83-c7ac8ef545df\") "
Aug 02 10:46:22.478370 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.478217    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9233794d-93fe-40f2-bd83-c7ac8ef545df-webhook-cert\") pod \"9233794d-93fe-40f2-bd83-c7ac8ef545df\" (UID: \"9233794d-93fe-40f2-bd83-c7ac8ef545df\") "
Aug 02 10:46:22.480450 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.480419    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9233794d-93fe-40f2-bd83-c7ac8ef545df-kube-api-access-h7496" (OuterVolumeSpecName: "kube-api-access-h7496") pod "9233794d-93fe-40f2-bd83-c7ac8ef545df" (UID: "9233794d-93fe-40f2-bd83-c7ac8ef545df"). InnerVolumeSpecName "kube-api-access-h7496". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:46:22.480561 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.480473    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9233794d-93fe-40f2-bd83-c7ac8ef545df-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9233794d-93fe-40f2-bd83-c7ac8ef545df" (UID: "9233794d-93fe-40f2-bd83-c7ac8ef545df"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:46:22.579517 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.579479    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9233794d-93fe-40f2-bd83-c7ac8ef545df-webhook-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:46:22.579517 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:22.579511    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7496\" (UniqueName: \"kubernetes.io/projected/9233794d-93fe-40f2-bd83-c7ac8ef545df-kube-api-access-h7496\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:46:23.104673 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:23.104637    2576 generic.go:356] "Generic (PLEG): container finished" podID="9233794d-93fe-40f2-bd83-c7ac8ef545df" containerID="ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2" exitCode=0
Aug 02 10:46:23.105176 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:23.104686    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-644f8d959b-5fkff"
Aug 02 10:46:23.105176 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:23.104728    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-644f8d959b-5fkff" event={"ID":"9233794d-93fe-40f2-bd83-c7ac8ef545df","Type":"ContainerDied","Data":"ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2"}
Aug 02 10:46:23.105176 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:23.104778    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-644f8d959b-5fkff" event={"ID":"9233794d-93fe-40f2-bd83-c7ac8ef545df","Type":"ContainerDied","Data":"3cd22ce63bef2efa4957378d4da12e83f79f2acfca764b4ca902d28ace936c80"}
Aug 02 10:46:23.105176 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:23.104799    2576 scope.go:117] "RemoveContainer" containerID="ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2"
Aug 02 10:46:23.113699 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:23.113583    2576 scope.go:117] "RemoveContainer" containerID="ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2"
Aug 02 10:46:23.113864 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:46:23.113847    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2\": container with ID starting with ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2 not found: ID does not exist" containerID="ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2"
Aug 02 10:46:23.113914 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:23.113873    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2"} err="failed to get container status \"ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2\": rpc error: code = NotFound desc = could not find container \"ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2\": container with ID starting with ecde1fc28e2403030ad2a554fccc0e4401e2ff555cc5811f74ca464376587ff2 not found: ID does not exist"
Aug 02 10:46:23.124994 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:23.124974    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-644f8d959b-5fkff"]
Aug 02 10:46:23.128762 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:23.128741    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-644f8d959b-5fkff"]
Aug 02 10:46:24.950075 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:24.950044    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9233794d-93fe-40f2-bd83-c7ac8ef545df" path="/var/lib/kubelet/pods/9233794d-93fe-40f2-bd83-c7ac8ef545df/volumes"
Aug 02 10:46:43.628246 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.628162    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-988754455-jhkss"]
Aug 02 10:46:43.628774 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.628542    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9233794d-93fe-40f2-bd83-c7ac8ef545df" containerName="manager"
Aug 02 10:46:43.628774 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.628553    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9233794d-93fe-40f2-bd83-c7ac8ef545df" containerName="manager"
Aug 02 10:46:43.628774 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.628613    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9233794d-93fe-40f2-bd83-c7ac8ef545df" containerName="manager"
Aug 02 10:46:43.631675 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.631656    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.634417 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.634384    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Aug 02 10:46:43.634529 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.634444    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kddq8\""
Aug 02 10:46:43.643127 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.643092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-988754455-jhkss"]
Aug 02 10:46:43.750511 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.750479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k92wx\" (UniqueName: \"kubernetes.io/projected/c719e60a-3c18-4055-8dea-380a13657702-kube-api-access-k92wx\") pod \"maas-api-988754455-jhkss\" (UID: \"c719e60a-3c18-4055-8dea-380a13657702\") " pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.750716 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.750529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c719e60a-3c18-4055-8dea-380a13657702-maas-api-tls\") pod \"maas-api-988754455-jhkss\" (UID: \"c719e60a-3c18-4055-8dea-380a13657702\") " pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.750716 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.750633    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c719e60a-3c18-4055-8dea-380a13657702-openshift-service-ca\") pod \"maas-api-988754455-jhkss\" (UID: \"c719e60a-3c18-4055-8dea-380a13657702\") " pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.851565 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.851530    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c719e60a-3c18-4055-8dea-380a13657702-openshift-service-ca\") pod \"maas-api-988754455-jhkss\" (UID: \"c719e60a-3c18-4055-8dea-380a13657702\") " pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.851720 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.851590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k92wx\" (UniqueName: \"kubernetes.io/projected/c719e60a-3c18-4055-8dea-380a13657702-kube-api-access-k92wx\") pod \"maas-api-988754455-jhkss\" (UID: \"c719e60a-3c18-4055-8dea-380a13657702\") " pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.851720 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.851640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c719e60a-3c18-4055-8dea-380a13657702-maas-api-tls\") pod \"maas-api-988754455-jhkss\" (UID: \"c719e60a-3c18-4055-8dea-380a13657702\") " pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.852361 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.852321    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c719e60a-3c18-4055-8dea-380a13657702-openshift-service-ca\") pod \"maas-api-988754455-jhkss\" (UID: \"c719e60a-3c18-4055-8dea-380a13657702\") " pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.854460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.854440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c719e60a-3c18-4055-8dea-380a13657702-maas-api-tls\") pod \"maas-api-988754455-jhkss\" (UID: \"c719e60a-3c18-4055-8dea-380a13657702\") " pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.860651 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.860630    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k92wx\" (UniqueName: \"kubernetes.io/projected/c719e60a-3c18-4055-8dea-380a13657702-kube-api-access-k92wx\") pod \"maas-api-988754455-jhkss\" (UID: \"c719e60a-3c18-4055-8dea-380a13657702\") " pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:43.943504 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:43.943478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:44.098321 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.098290    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-988754455-jhkss"]
Aug 02 10:46:44.100349 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:46:44.100318    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc719e60a_3c18_4055_8dea_380a13657702.slice/crio-f6aab82e345795ecb0453f05388e1dbb5cd8abb97e7302b929d49b413be44e9a WatchSource:0}: Error finding container f6aab82e345795ecb0453f05388e1dbb5cd8abb97e7302b929d49b413be44e9a: Status 404 returned error can't find the container with id f6aab82e345795ecb0453f05388e1dbb5cd8abb97e7302b929d49b413be44e9a
Aug 02 10:46:44.120815 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.120785    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"]
Aug 02 10:46:44.125344 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.125311    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:44.128050 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.128018    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 02 10:46:44.138680 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.138654    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"]
Aug 02 10:46:44.154252 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.154226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc491347-ccc2-4e2f-8d7f-a716008c5279-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-sgwqw\" (UID: \"fc491347-ccc2-4e2f-8d7f-a716008c5279\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:44.154385 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.154272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpkbx\" (UniqueName: \"kubernetes.io/projected/fc491347-ccc2-4e2f-8d7f-a716008c5279-kube-api-access-mpkbx\") pod \"payload-pre-processing-687dbb6cf6-sgwqw\" (UID: \"fc491347-ccc2-4e2f-8d7f-a716008c5279\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:44.167655 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.167623    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-w6bc7"]
Aug 02 10:46:44.170859 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.170837    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:44.175479 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.175454    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dwnz8\""
Aug 02 10:46:44.187683 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.187646    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-w6bc7"]
Aug 02 10:46:44.218885 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.218807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-988754455-jhkss" event={"ID":"c719e60a-3c18-4055-8dea-380a13657702","Type":"ContainerStarted","Data":"f6aab82e345795ecb0453f05388e1dbb5cd8abb97e7302b929d49b413be44e9a"}
Aug 02 10:46:44.254766 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.254727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpkbx\" (UniqueName: \"kubernetes.io/projected/fc491347-ccc2-4e2f-8d7f-a716008c5279-kube-api-access-mpkbx\") pod \"payload-pre-processing-687dbb6cf6-sgwqw\" (UID: \"fc491347-ccc2-4e2f-8d7f-a716008c5279\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:44.254934 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.254796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1787e6d1-3ebf-4824-9540-88d0b2022fd6-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-w6bc7\" (UID: \"1787e6d1-3ebf-4824-9540-88d0b2022fd6\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:44.254934 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.254912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lncng\" (UniqueName: \"kubernetes.io/projected/1787e6d1-3ebf-4824-9540-88d0b2022fd6-kube-api-access-lncng\") pod \"payload-processing-5cc6cd444b-w6bc7\" (UID: \"1787e6d1-3ebf-4824-9540-88d0b2022fd6\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:44.255068 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.254988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc491347-ccc2-4e2f-8d7f-a716008c5279-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-sgwqw\" (UID: \"fc491347-ccc2-4e2f-8d7f-a716008c5279\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:44.255695 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.255676    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc491347-ccc2-4e2f-8d7f-a716008c5279-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-sgwqw\" (UID: \"fc491347-ccc2-4e2f-8d7f-a716008c5279\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:44.265832 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.265805    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpkbx\" (UniqueName: \"kubernetes.io/projected/fc491347-ccc2-4e2f-8d7f-a716008c5279-kube-api-access-mpkbx\") pod \"payload-pre-processing-687dbb6cf6-sgwqw\" (UID: \"fc491347-ccc2-4e2f-8d7f-a716008c5279\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:44.355588 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.355556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1787e6d1-3ebf-4824-9540-88d0b2022fd6-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-w6bc7\" (UID: \"1787e6d1-3ebf-4824-9540-88d0b2022fd6\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:44.355754 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.355600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lncng\" (UniqueName: \"kubernetes.io/projected/1787e6d1-3ebf-4824-9540-88d0b2022fd6-kube-api-access-lncng\") pod \"payload-processing-5cc6cd444b-w6bc7\" (UID: \"1787e6d1-3ebf-4824-9540-88d0b2022fd6\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:44.356266 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.356243    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1787e6d1-3ebf-4824-9540-88d0b2022fd6-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-w6bc7\" (UID: \"1787e6d1-3ebf-4824-9540-88d0b2022fd6\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:44.364732 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.364702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lncng\" (UniqueName: \"kubernetes.io/projected/1787e6d1-3ebf-4824-9540-88d0b2022fd6-kube-api-access-lncng\") pod \"payload-processing-5cc6cd444b-w6bc7\" (UID: \"1787e6d1-3ebf-4824-9540-88d0b2022fd6\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:44.437203 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.437170    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:44.484869 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.484780    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:44.596690 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.596661    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"]
Aug 02 10:46:44.597942 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:46:44.597906    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc491347_ccc2_4e2f_8d7f_a716008c5279.slice/crio-7e5bd79f3d4ec63ef5e5d6a56822fc87b76ea7aa6b4015d06196acc975fe43ac WatchSource:0}: Error finding container 7e5bd79f3d4ec63ef5e5d6a56822fc87b76ea7aa6b4015d06196acc975fe43ac: Status 404 returned error can't find the container with id 7e5bd79f3d4ec63ef5e5d6a56822fc87b76ea7aa6b4015d06196acc975fe43ac
Aug 02 10:46:44.673136 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:44.673106    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-w6bc7"]
Aug 02 10:46:44.674550 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:46:44.674519    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1787e6d1_3ebf_4824_9540_88d0b2022fd6.slice/crio-cacaa67a6b70545eadf7d2af15a45ca866a622f9341a395ea21e788c8a391871 WatchSource:0}: Error finding container cacaa67a6b70545eadf7d2af15a45ca866a622f9341a395ea21e788c8a391871: Status 404 returned error can't find the container with id cacaa67a6b70545eadf7d2af15a45ca866a622f9341a395ea21e788c8a391871
Aug 02 10:46:45.228238 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:45.228145    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7" event={"ID":"1787e6d1-3ebf-4824-9540-88d0b2022fd6","Type":"ContainerStarted","Data":"cacaa67a6b70545eadf7d2af15a45ca866a622f9341a395ea21e788c8a391871"}
Aug 02 10:46:45.231304 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:45.231205    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw" event={"ID":"fc491347-ccc2-4e2f-8d7f-a716008c5279","Type":"ContainerStarted","Data":"7e5bd79f3d4ec63ef5e5d6a56822fc87b76ea7aa6b4015d06196acc975fe43ac"}
Aug 02 10:46:47.247966 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:47.247924    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-988754455-jhkss" event={"ID":"c719e60a-3c18-4055-8dea-380a13657702","Type":"ContainerStarted","Data":"2e8f1fba44053e18c404952198d086c494a7f6373679f54ec3839dd1072e8282"}
Aug 02 10:46:47.248420 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:47.248065    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:47.249492 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:47.249436    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7" event={"ID":"1787e6d1-3ebf-4824-9540-88d0b2022fd6","Type":"ContainerStarted","Data":"7c2d85be15da0c309a00582172f98429307ec01a757cb472659a3413e8bf724d"}
Aug 02 10:46:47.249492 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:47.249484    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:47.250845 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:47.250825    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw" event={"ID":"fc491347-ccc2-4e2f-8d7f-a716008c5279","Type":"ContainerStarted","Data":"f794dec46f03cc80b59ba751abd81c15e5463292ead39b9c7a173f72710415d6"}
Aug 02 10:46:47.250954 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:47.250931    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:47.267166 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:47.267109    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-988754455-jhkss" podStartSLOduration=1.907449588 podStartE2EDuration="4.267095589s" totalImagesPullingTime="2.359646001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:46:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:46:47.265383581 +0000 UTC m=+808.795966235" watchObservedRunningTime="2026-08-02 10:46:47.267095589 +0000 UTC m=+808.797678218"
Aug 02 10:46:47.283080 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:47.283017    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw" podStartSLOduration=0.863483757 podStartE2EDuration="3.283003686s" totalImagesPullingTime="2.419519929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:46:47.280953923 +0000 UTC m=+808.811536554" watchObservedRunningTime="2026-08-02 10:46:47.283003686 +0000 UTC m=+808.813586315"
Aug 02 10:46:47.298720 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:47.298664    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7" podStartSLOduration=0.948977436 podStartE2EDuration="3.298645622s" totalImagesPullingTime="2.349668186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:46:47.296530553 +0000 UTC m=+808.827113201" watchObservedRunningTime="2026-08-02 10:46:47.298645622 +0000 UTC m=+808.829228253"
Aug 02 10:46:53.261996 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:53.261921    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-988754455-jhkss"
Aug 02 10:46:53.285913 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:53.285879    2576 generic.go:356] "Generic (PLEG): container finished" podID="8b5b18a4-54de-4def-ae35-0cf0dab8deae" containerID="695f624951a1005e96565aa5725a81405ab538bc3858e0074639f83378b7edae" exitCode=0
Aug 02 10:46:53.286076 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:53.285953    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" event={"ID":"8b5b18a4-54de-4def-ae35-0cf0dab8deae","Type":"ContainerDied","Data":"695f624951a1005e96565aa5725a81405ab538bc3858e0074639f83378b7edae"}
Aug 02 10:46:53.286314 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:53.286301    2576 scope.go:117] "RemoveContainer" containerID="695f624951a1005e96565aa5725a81405ab538bc3858e0074639f83378b7edae"
Aug 02 10:46:53.287265 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:53.287239    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 10:46:53.287326 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:53.287302    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 10:46:53.287365 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:53.287333    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 10:46:54.297270 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:54.297228    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" event={"ID":"8b5b18a4-54de-4def-ae35-0cf0dab8deae","Type":"ContainerStarted","Data":"122708c0e1e7b051f060a12727536f0f3f02fbe7d2278b7b513b7b413222fa8b"}
Aug 02 10:46:54.313258 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:54.313218    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:46:54.313460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:54.313391    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnqf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Aug 02 10:46:54.313460 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:54.313445    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" podUID="8b5b18a4-54de-4def-ae35-0cf0dab8deae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Aug 02 10:46:55.309055 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:55.308991    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnqf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Aug 02 10:46:55.309530 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:55.309073    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" podUID="8b5b18a4-54de-4def-ae35-0cf0dab8deae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Aug 02 10:46:56.311573 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:56.311117    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnqf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Aug 02 10:46:56.311573 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:56.311192    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" podUID="8b5b18a4-54de-4def-ae35-0cf0dab8deae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Aug 02 10:46:57.308985 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:57.308947    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnqf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Aug 02 10:46:57.309320 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:57.309288    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" podUID="8b5b18a4-54de-4def-ae35-0cf0dab8deae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Aug 02 10:46:58.258251 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:58.258217    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-w6bc7"
Aug 02 10:46:58.258712 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:58.258542    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sgwqw"
Aug 02 10:46:58.310345 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:58.310310    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dnqf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Aug 02 10:46:58.310627 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:58.310587    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6" podUID="8b5b18a4-54de-4def-ae35-0cf0dab8deae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Aug 02 10:46:59.359249 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:59.359215    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:46:59.359736 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:59.359649    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:46:59.360587 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:46:59.360568    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dnqf6"
Aug 02 10:47:08.260741 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.260705    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"]
Aug 02 10:47:08.265473 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.265454    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.269455 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.269432    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 02 10:47:08.270687 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.270662    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Aug 02 10:47:08.270811 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.270677    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 02 10:47:08.270979 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.270791    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fkv5b\""
Aug 02 10:47:08.274452 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.274432    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"]
Aug 02 10:47:08.392040 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.392002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.392206 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.392114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpmcr\" (UniqueName: \"kubernetes.io/projected/0a081475-63ee-4ec9-9e90-26743e0b15ca-kube-api-access-wpmcr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.392250 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.392213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.392286 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.392275    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a081475-63ee-4ec9-9e90-26743e0b15ca-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.392321 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.392305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.392355 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.392343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.493337 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.493306    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.493549 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.493345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.493549 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.493367    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.493549 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.493406    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpmcr\" (UniqueName: \"kubernetes.io/projected/0a081475-63ee-4ec9-9e90-26743e0b15ca-kube-api-access-wpmcr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.493549 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.493475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.493549 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.493522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a081475-63ee-4ec9-9e90-26743e0b15ca-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.493843 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.493733    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.493843 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.493753    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.493961 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.493939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.495771 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.495749    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a081475-63ee-4ec9-9e90-26743e0b15ca-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.496191 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.496169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a081475-63ee-4ec9-9e90-26743e0b15ca-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.502940 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.502917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpmcr\" (UniqueName: \"kubernetes.io/projected/0a081475-63ee-4ec9-9e90-26743e0b15ca-kube-api-access-wpmcr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b86l7\" (UID: \"0a081475-63ee-4ec9-9e90-26743e0b15ca\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.577281 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.577220    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:08.745445 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:08.745421    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"]
Aug 02 10:47:08.746987 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:47:08.746961    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a081475_63ee_4ec9_9e90_26743e0b15ca.slice/crio-e5e371cca1a28667d44747ed19c6449167f46a92e3d4aa854013bc15ba12db72 WatchSource:0}: Error finding container e5e371cca1a28667d44747ed19c6449167f46a92e3d4aa854013bc15ba12db72: Status 404 returned error can't find the container with id e5e371cca1a28667d44747ed19c6449167f46a92e3d4aa854013bc15ba12db72
Aug 02 10:47:09.393616 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:09.393576    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7" event={"ID":"0a081475-63ee-4ec9-9e90-26743e0b15ca","Type":"ContainerStarted","Data":"e5e371cca1a28667d44747ed19c6449167f46a92e3d4aa854013bc15ba12db72"}
Aug 02 10:47:10.400958 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:10.400919    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7" event={"ID":"0a081475-63ee-4ec9-9e90-26743e0b15ca","Type":"ContainerStarted","Data":"5e8e2fefe182db712aea272bb2e2ea37163f6b480d6e71865d3c843bf56de153"}
Aug 02 10:47:10.436219 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:10.436175    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7" podStartSLOduration=1.4310398420000001 podStartE2EDuration="2.436158336s" totalImagesPullingTime="1.005118494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:47:10.435547223 +0000 UTC m=+831.966129852" watchObservedRunningTime="2026-08-02 10:47:10.436158336 +0000 UTC m=+831.966740966"
Aug 02 10:47:18.577612 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:18.577578    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:18.578136 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:18.577664    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:18.590202 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:18.590180    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:19.463008 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:19.462981    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b86l7"
Aug 02 10:47:41.054424 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.054393    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"]
Aug 02 10:47:41.057440 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.057423    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.060198 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.060178    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Aug 02 10:47:41.067066 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.067045    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"]
Aug 02 10:47:41.174849 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.174816    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.175007 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.174883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.175007 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.174907    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnxr8\" (UniqueName: \"kubernetes.io/projected/1387258c-8616-4cfd-a6a8-c805b461c20c-kube-api-access-dnxr8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.175007 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.174937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.175208 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.175045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1387258c-8616-4cfd-a6a8-c805b461c20c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.175208 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.175110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.275849 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.275814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.276006 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.275860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1387258c-8616-4cfd-a6a8-c805b461c20c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.276006 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.275972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.276133 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.276018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.276133 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.276115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.276234 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.276136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnxr8\" (UniqueName: \"kubernetes.io/projected/1387258c-8616-4cfd-a6a8-c805b461c20c-kube-api-access-dnxr8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.276335 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.276311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.276426 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.276406    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.276490 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.276458    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.278350 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.278329    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1387258c-8616-4cfd-a6a8-c805b461c20c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.278617 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.278599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1387258c-8616-4cfd-a6a8-c805b461c20c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.287411 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.287377    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnxr8\" (UniqueName: \"kubernetes.io/projected/1387258c-8616-4cfd-a6a8-c805b461c20c-kube-api-access-dnxr8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p9wdk\" (UID: \"1387258c-8616-4cfd-a6a8-c805b461c20c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.368860 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.368777    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:41.512885 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.512802    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"]
Aug 02 10:47:41.515777 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:47:41.515749    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1387258c_8616_4cfd_a6a8_c805b461c20c.slice/crio-ab90b7061ce7fbb270cd586f2e539160fea50ebd27516cc81986a17ae982724f WatchSource:0}: Error finding container ab90b7061ce7fbb270cd586f2e539160fea50ebd27516cc81986a17ae982724f: Status 404 returned error can't find the container with id ab90b7061ce7fbb270cd586f2e539160fea50ebd27516cc81986a17ae982724f
Aug 02 10:47:41.572215 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:41.572186    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk" event={"ID":"1387258c-8616-4cfd-a6a8-c805b461c20c","Type":"ContainerStarted","Data":"ab90b7061ce7fbb270cd586f2e539160fea50ebd27516cc81986a17ae982724f"}
Aug 02 10:47:42.579369 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:42.579337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk" event={"ID":"1387258c-8616-4cfd-a6a8-c805b461c20c","Type":"ContainerStarted","Data":"5ba6bebb394703beff5072dc8c7cea99d60b17b939d5a7ecb36ad3f526337afd"}
Aug 02 10:47:42.598627 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:42.598577    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk" podStartSLOduration=1.407479103 podStartE2EDuration="1.598562026s" totalImagesPullingTime="191.082923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:47:42.597351433 +0000 UTC m=+864.127934076" watchObservedRunningTime="2026-08-02 10:47:42.598562026 +0000 UTC m=+864.129145043"
Aug 02 10:47:51.369543 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:51.369512    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:51.370113 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:51.369580    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:51.382082 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:51.382063    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:47:51.643759 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:47:51.643666    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p9wdk"
Aug 02 10:48:18.877150 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:48:18.877124    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:48:18.877613 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:48:18.877580    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:49:34.160894 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:34.160863    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-2vhqv"]
Aug 02 10:49:34.161402 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:34.161114    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57f59cbf46-2vhqv" podUID="585a7939-76a1-428b-8546-5eca4a3ddeba" containerName="manager" containerID="cri-o://1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798" gracePeriod=30
Aug 02 10:49:34.426564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:34.426538    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:49:34.563800 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:34.563764    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/585a7939-76a1-428b-8546-5eca4a3ddeba-webhook-cert\") pod \"585a7939-76a1-428b-8546-5eca4a3ddeba\" (UID: \"585a7939-76a1-428b-8546-5eca4a3ddeba\") "
Aug 02 10:49:34.563969 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:34.563891    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfzhn\" (UniqueName: \"kubernetes.io/projected/585a7939-76a1-428b-8546-5eca4a3ddeba-kube-api-access-nfzhn\") pod \"585a7939-76a1-428b-8546-5eca4a3ddeba\" (UID: \"585a7939-76a1-428b-8546-5eca4a3ddeba\") "
Aug 02 10:49:34.565974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:34.565941    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/585a7939-76a1-428b-8546-5eca4a3ddeba-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "585a7939-76a1-428b-8546-5eca4a3ddeba" (UID: "585a7939-76a1-428b-8546-5eca4a3ddeba"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:49:34.566114 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:34.566000    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/585a7939-76a1-428b-8546-5eca4a3ddeba-kube-api-access-nfzhn" (OuterVolumeSpecName: "kube-api-access-nfzhn") pod "585a7939-76a1-428b-8546-5eca4a3ddeba" (UID: "585a7939-76a1-428b-8546-5eca4a3ddeba"). InnerVolumeSpecName "kube-api-access-nfzhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:49:34.665226 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:34.665152    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/585a7939-76a1-428b-8546-5eca4a3ddeba-webhook-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:49:34.665226 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:34.665180    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfzhn\" (UniqueName: \"kubernetes.io/projected/585a7939-76a1-428b-8546-5eca4a3ddeba-kube-api-access-nfzhn\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:49:35.219665 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:35.219577    2576 generic.go:356] "Generic (PLEG): container finished" podID="585a7939-76a1-428b-8546-5eca4a3ddeba" containerID="1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798" exitCode=0
Aug 02 10:49:35.219665 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:35.219623    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-2vhqv"
Aug 02 10:49:35.219665 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:35.219623    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-2vhqv" event={"ID":"585a7939-76a1-428b-8546-5eca4a3ddeba","Type":"ContainerDied","Data":"1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798"}
Aug 02 10:49:35.220235 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:35.219684    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-2vhqv" event={"ID":"585a7939-76a1-428b-8546-5eca4a3ddeba","Type":"ContainerDied","Data":"e4d15f78534a14dd59ea9ff7f6f4238b3081778f85cf61f0e425404785d5ba94"}
Aug 02 10:49:35.220235 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:35.219706    2576 scope.go:117] "RemoveContainer" containerID="1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798"
Aug 02 10:49:35.228553 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:35.228533    2576 scope.go:117] "RemoveContainer" containerID="1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798"
Aug 02 10:49:35.228805 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:49:35.228786    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798\": container with ID starting with 1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798 not found: ID does not exist" containerID="1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798"
Aug 02 10:49:35.228854 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:35.228812    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798"} err="failed to get container status \"1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798\": rpc error: code = NotFound desc = could not find container \"1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798\": container with ID starting with 1bf44da204bf1431ab66031707b937e2f306f27b21a4c8a9462d7ad3dcdd6798 not found: ID does not exist"
Aug 02 10:49:35.242315 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:35.242292    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-2vhqv"]
Aug 02 10:49:35.251116 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:35.251094    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-2vhqv"]
Aug 02 10:49:36.951072 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:49:36.951014    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585a7939-76a1-428b-8546-5eca4a3ddeba" path="/var/lib/kubelet/pods/585a7939-76a1-428b-8546-5eca4a3ddeba/volumes"
Aug 02 10:50:36.070206 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.070177    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57f59cbf46-fq8ml"]
Aug 02 10:50:36.070593 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.070551    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585a7939-76a1-428b-8546-5eca4a3ddeba" containerName="manager"
Aug 02 10:50:36.070593 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.070562    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="585a7939-76a1-428b-8546-5eca4a3ddeba" containerName="manager"
Aug 02 10:50:36.070665 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.070632    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="585a7939-76a1-428b-8546-5eca4a3ddeba" containerName="manager"
Aug 02 10:50:36.074806 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.074789    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:36.077798 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.077779    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-76z9v\""
Aug 02 10:50:36.078810 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.078790    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 02 10:50:36.078915 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.078840    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 02 10:50:36.086093 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.086065    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-fq8ml"]
Aug 02 10:50:36.186387 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.186360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kctjh\" (UniqueName: \"kubernetes.io/projected/a4943093-e4c1-45dd-8862-0af1e0f6d106-kube-api-access-kctjh\") pod \"maas-controller-57f59cbf46-fq8ml\" (UID: \"a4943093-e4c1-45dd-8862-0af1e0f6d106\") " pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:36.186551 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.186407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4943093-e4c1-45dd-8862-0af1e0f6d106-webhook-cert\") pod \"maas-controller-57f59cbf46-fq8ml\" (UID: \"a4943093-e4c1-45dd-8862-0af1e0f6d106\") " pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:36.286863 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.286834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kctjh\" (UniqueName: \"kubernetes.io/projected/a4943093-e4c1-45dd-8862-0af1e0f6d106-kube-api-access-kctjh\") pod \"maas-controller-57f59cbf46-fq8ml\" (UID: \"a4943093-e4c1-45dd-8862-0af1e0f6d106\") " pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:36.287018 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.286877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4943093-e4c1-45dd-8862-0af1e0f6d106-webhook-cert\") pod \"maas-controller-57f59cbf46-fq8ml\" (UID: \"a4943093-e4c1-45dd-8862-0af1e0f6d106\") " pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:36.289216 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.289192    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4943093-e4c1-45dd-8862-0af1e0f6d106-webhook-cert\") pod \"maas-controller-57f59cbf46-fq8ml\" (UID: \"a4943093-e4c1-45dd-8862-0af1e0f6d106\") " pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:36.297211 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.297176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kctjh\" (UniqueName: \"kubernetes.io/projected/a4943093-e4c1-45dd-8862-0af1e0f6d106-kube-api-access-kctjh\") pod \"maas-controller-57f59cbf46-fq8ml\" (UID: \"a4943093-e4c1-45dd-8862-0af1e0f6d106\") " pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:36.385883 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.385797    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:36.525677 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.525577    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-fq8ml"]
Aug 02 10:50:36.527518 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:50:36.527458    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4943093_e4c1_45dd_8862_0af1e0f6d106.slice/crio-599a116c3422396bf0064bb18847cd6468f61c33729944ae192ad5c6ff60c0b4 WatchSource:0}: Error finding container 599a116c3422396bf0064bb18847cd6468f61c33729944ae192ad5c6ff60c0b4: Status 404 returned error can't find the container with id 599a116c3422396bf0064bb18847cd6468f61c33729944ae192ad5c6ff60c0b4
Aug 02 10:50:36.529341 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.529318    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 02 10:50:36.550205 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:36.550178    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-fq8ml" event={"ID":"a4943093-e4c1-45dd-8862-0af1e0f6d106","Type":"ContainerStarted","Data":"599a116c3422396bf0064bb18847cd6468f61c33729944ae192ad5c6ff60c0b4"}
Aug 02 10:50:37.557352 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:37.557316    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-fq8ml" event={"ID":"a4943093-e4c1-45dd-8862-0af1e0f6d106","Type":"ContainerStarted","Data":"03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0"}
Aug 02 10:50:37.557726 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:37.557461    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:37.581780 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:37.581723    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57f59cbf46-fq8ml" podStartSLOduration=1.160363857 podStartE2EDuration="1.581707777s" totalImagesPullingTime="421.34392ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:50:37.578636342 +0000 UTC m=+1039.109218970" watchObservedRunningTime="2026-08-02 10:50:37.581707777 +0000 UTC m=+1039.112290405"
Aug 02 10:50:48.564738 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:48.564711    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:53.244841 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.244805    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-fq8ml"]
Aug 02 10:50:53.245284 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.245000    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57f59cbf46-fq8ml" podUID="a4943093-e4c1-45dd-8862-0af1e0f6d106" containerName="manager" containerID="cri-o://03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0" gracePeriod=30
Aug 02 10:50:53.497741 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.497692    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:53.633664 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.633629    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4943093-e4c1-45dd-8862-0af1e0f6d106-webhook-cert\") pod \"a4943093-e4c1-45dd-8862-0af1e0f6d106\" (UID: \"a4943093-e4c1-45dd-8862-0af1e0f6d106\") "
Aug 02 10:50:53.633862 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.633731    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kctjh\" (UniqueName: \"kubernetes.io/projected/a4943093-e4c1-45dd-8862-0af1e0f6d106-kube-api-access-kctjh\") pod \"a4943093-e4c1-45dd-8862-0af1e0f6d106\" (UID: \"a4943093-e4c1-45dd-8862-0af1e0f6d106\") "
Aug 02 10:50:53.635787 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.635759    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4943093-e4c1-45dd-8862-0af1e0f6d106-kube-api-access-kctjh" (OuterVolumeSpecName: "kube-api-access-kctjh") pod "a4943093-e4c1-45dd-8862-0af1e0f6d106" (UID: "a4943093-e4c1-45dd-8862-0af1e0f6d106"). InnerVolumeSpecName "kube-api-access-kctjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:50:53.635896 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.635846    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4943093-e4c1-45dd-8862-0af1e0f6d106-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a4943093-e4c1-45dd-8862-0af1e0f6d106" (UID: "a4943093-e4c1-45dd-8862-0af1e0f6d106"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 10:50:53.645015 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.644987    2576 generic.go:356] "Generic (PLEG): container finished" podID="a4943093-e4c1-45dd-8862-0af1e0f6d106" containerID="03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0" exitCode=0
Aug 02 10:50:53.645143 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.645043    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-fq8ml"
Aug 02 10:50:53.645143 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.645070    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-fq8ml" event={"ID":"a4943093-e4c1-45dd-8862-0af1e0f6d106","Type":"ContainerDied","Data":"03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0"}
Aug 02 10:50:53.645143 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.645134    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-fq8ml" event={"ID":"a4943093-e4c1-45dd-8862-0af1e0f6d106","Type":"ContainerDied","Data":"599a116c3422396bf0064bb18847cd6468f61c33729944ae192ad5c6ff60c0b4"}
Aug 02 10:50:53.645313 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.645164    2576 scope.go:117] "RemoveContainer" containerID="03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0"
Aug 02 10:50:53.657135 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.657117    2576 scope.go:117] "RemoveContainer" containerID="03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0"
Aug 02 10:50:53.657376 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:50:53.657360    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0\": container with ID starting with 03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0 not found: ID does not exist" containerID="03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0"
Aug 02 10:50:53.657432 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.657383    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0"} err="failed to get container status \"03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0\": rpc error: code = NotFound desc = could not find container \"03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0\": container with ID starting with 03b7b886feab6f24f7066a2251170425308dcb79f11d0ff010de7e4a952964d0 not found: ID does not exist"
Aug 02 10:50:53.670974 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.670942    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-fq8ml"]
Aug 02 10:50:53.681646 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.678175    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-fq8ml"]
Aug 02 10:50:53.734662 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.734639    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kctjh\" (UniqueName: \"kubernetes.io/projected/a4943093-e4c1-45dd-8862-0af1e0f6d106-kube-api-access-kctjh\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:50:53.734662 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:53.734662    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4943093-e4c1-45dd-8862-0af1e0f6d106-webhook-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:50:54.462094 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.462052    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57f59cbf46-tsjxw"]
Aug 02 10:50:54.462494 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.462481    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4943093-e4c1-45dd-8862-0af1e0f6d106" containerName="manager"
Aug 02 10:50:54.462537 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.462496    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4943093-e4c1-45dd-8862-0af1e0f6d106" containerName="manager"
Aug 02 10:50:54.462579 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.462570    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4943093-e4c1-45dd-8862-0af1e0f6d106" containerName="manager"
Aug 02 10:50:54.466816 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.466799    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:50:54.470926 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.470899    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 02 10:50:54.471075 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.470943    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-76z9v\""
Aug 02 10:50:54.471075 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.470939    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 02 10:50:54.475386 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.475365    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-tsjxw"]
Aug 02 10:50:54.643682 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.643647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7lgw\" (UniqueName: \"kubernetes.io/projected/a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65-kube-api-access-l7lgw\") pod \"maas-controller-57f59cbf46-tsjxw\" (UID: \"a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65\") " pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:50:54.643841 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.643696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65-webhook-cert\") pod \"maas-controller-57f59cbf46-tsjxw\" (UID: \"a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65\") " pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:50:54.744858 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.744782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7lgw\" (UniqueName: \"kubernetes.io/projected/a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65-kube-api-access-l7lgw\") pod \"maas-controller-57f59cbf46-tsjxw\" (UID: \"a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65\") " pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:50:54.744858 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.744837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65-webhook-cert\") pod \"maas-controller-57f59cbf46-tsjxw\" (UID: \"a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65\") " pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:50:54.747517 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.747497    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65-webhook-cert\") pod \"maas-controller-57f59cbf46-tsjxw\" (UID: \"a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65\") " pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:50:54.756384 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.756362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7lgw\" (UniqueName: \"kubernetes.io/projected/a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65-kube-api-access-l7lgw\") pod \"maas-controller-57f59cbf46-tsjxw\" (UID: \"a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65\") " pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:50:54.777957 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.777935    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:50:54.914238 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.914214    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f59cbf46-tsjxw"]
Aug 02 10:50:54.915706 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:50:54.915679    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8db5efb_5a93_4fe6_a2bc_7a1cb6526a65.slice/crio-256f226eb6a4f76911f650ca36b3d22197a42b0ef4e91da752c756bf287a6436 WatchSource:0}: Error finding container 256f226eb6a4f76911f650ca36b3d22197a42b0ef4e91da752c756bf287a6436: Status 404 returned error can't find the container with id 256f226eb6a4f76911f650ca36b3d22197a42b0ef4e91da752c756bf287a6436
Aug 02 10:50:54.951309 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:54.951283    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4943093-e4c1-45dd-8862-0af1e0f6d106" path="/var/lib/kubelet/pods/a4943093-e4c1-45dd-8862-0af1e0f6d106/volumes"
Aug 02 10:50:55.658983 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:55.658950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-tsjxw" event={"ID":"a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65","Type":"ContainerStarted","Data":"6f7d5285e1fec725263a923a2b9a69d069dbc7124bf246cca689824685b3f8d2"}
Aug 02 10:50:55.659495 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:55.659005    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f59cbf46-tsjxw" event={"ID":"a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65","Type":"ContainerStarted","Data":"256f226eb6a4f76911f650ca36b3d22197a42b0ef4e91da752c756bf287a6436"}
Aug 02 10:50:55.659495 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:55.659164    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:50:55.679233 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:50:55.679184    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57f59cbf46-tsjxw" podStartSLOduration=1.351714972 podStartE2EDuration="1.679169887s" totalImagesPullingTime="327.454915ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:50:55.678804837 +0000 UTC m=+1057.209387487" watchObservedRunningTime="2026-08-02 10:50:55.679169887 +0000 UTC m=+1057.209752516"
Aug 02 10:51:06.667240 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:51:06.667203    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57f59cbf46-tsjxw"
Aug 02 10:53:18.912305 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:53:18.912278    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:53:18.914774 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:53:18.914062    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:57:18.801229 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:18.801194    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"]
Aug 02 10:57:18.801748 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:18.801391    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z" podUID="7cae154b-f9ba-432b-94eb-892de780dcf9" containerName="manager" containerID="cri-o://385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9" gracePeriod=10
Aug 02 10:57:19.057010 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.056953    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:57:19.092439 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.092412    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq78c\" (UniqueName: \"kubernetes.io/projected/7cae154b-f9ba-432b-94eb-892de780dcf9-kube-api-access-kq78c\") pod \"7cae154b-f9ba-432b-94eb-892de780dcf9\" (UID: \"7cae154b-f9ba-432b-94eb-892de780dcf9\") "
Aug 02 10:57:19.092629 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.092449    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae154b-f9ba-432b-94eb-892de780dcf9-extensions-socket-volume\") pod \"7cae154b-f9ba-432b-94eb-892de780dcf9\" (UID: \"7cae154b-f9ba-432b-94eb-892de780dcf9\") "
Aug 02 10:57:19.092852 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.092829    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cae154b-f9ba-432b-94eb-892de780dcf9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7cae154b-f9ba-432b-94eb-892de780dcf9" (UID: "7cae154b-f9ba-432b-94eb-892de780dcf9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 10:57:19.094717 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.094687    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cae154b-f9ba-432b-94eb-892de780dcf9-kube-api-access-kq78c" (OuterVolumeSpecName: "kube-api-access-kq78c") pod "7cae154b-f9ba-432b-94eb-892de780dcf9" (UID: "7cae154b-f9ba-432b-94eb-892de780dcf9"). InnerVolumeSpecName "kube-api-access-kq78c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 10:57:19.193129 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.193086    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq78c\" (UniqueName: \"kubernetes.io/projected/7cae154b-f9ba-432b-94eb-892de780dcf9-kube-api-access-kq78c\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:57:19.193129 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.193113    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae154b-f9ba-432b-94eb-892de780dcf9-extensions-socket-volume\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 10:57:19.789230 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.789200    2576 generic.go:356] "Generic (PLEG): container finished" podID="7cae154b-f9ba-432b-94eb-892de780dcf9" containerID="385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9" exitCode=0
Aug 02 10:57:19.789412 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.789246    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"
Aug 02 10:57:19.789412 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.789297    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z" event={"ID":"7cae154b-f9ba-432b-94eb-892de780dcf9","Type":"ContainerDied","Data":"385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9"}
Aug 02 10:57:19.789412 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.789342    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z" event={"ID":"7cae154b-f9ba-432b-94eb-892de780dcf9","Type":"ContainerDied","Data":"d5d09d06f829d6c5361b7f90e28316420393bbb50f22e61dcd0df3242a5c2616"}
Aug 02 10:57:19.789412 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.789363    2576 scope.go:117] "RemoveContainer" containerID="385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9"
Aug 02 10:57:19.800096 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.800074    2576 scope.go:117] "RemoveContainer" containerID="385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9"
Aug 02 10:57:19.800354 ip-10-0-135-193 kubenswrapper[2576]: E0802 10:57:19.800337    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9\": container with ID starting with 385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9 not found: ID does not exist" containerID="385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9"
Aug 02 10:57:19.800419 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.800365    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9"} err="failed to get container status \"385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9\": rpc error: code = NotFound desc = could not find container \"385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9\": container with ID starting with 385a26df59d86dfd588c4dabb627f988572e3ed4779a5484a1b5984fd0a4c8e9 not found: ID does not exist"
Aug 02 10:57:19.814212 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.814183    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"]
Aug 02 10:57:19.818075 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:19.818052    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w9z8z"]
Aug 02 10:57:20.950434 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:57:20.950400    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cae154b-f9ba-432b-94eb-892de780dcf9" path="/var/lib/kubelet/pods/7cae154b-f9ba-432b-94eb-892de780dcf9/volumes"
Aug 02 10:58:18.959088 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:18.958965    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:58:18.962893 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:18.962878    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 10:58:19.504566 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.504534    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"]
Aug 02 10:58:19.504984 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.504970    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cae154b-f9ba-432b-94eb-892de780dcf9" containerName="manager"
Aug 02 10:58:19.505062 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.504985    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cae154b-f9ba-432b-94eb-892de780dcf9" containerName="manager"
Aug 02 10:58:19.505105 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.505069    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cae154b-f9ba-432b-94eb-892de780dcf9" containerName="manager"
Aug 02 10:58:19.509206 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.509185    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 10:58:19.511811 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.511792    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fjhc6\""
Aug 02 10:58:19.520290 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.520264    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"]
Aug 02 10:58:19.608855 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.608830    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff210be6-dbd3-451f-a26f-ad1c213d599d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8z9m2\" (UID: \"ff210be6-dbd3-451f-a26f-ad1c213d599d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 10:58:19.609017 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.608867    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6z6h\" (UniqueName: \"kubernetes.io/projected/ff210be6-dbd3-451f-a26f-ad1c213d599d-kube-api-access-d6z6h\") pod \"kuadrant-operator-controller-manager-68cf589589-8z9m2\" (UID: \"ff210be6-dbd3-451f-a26f-ad1c213d599d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 10:58:19.709705 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.709676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff210be6-dbd3-451f-a26f-ad1c213d599d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8z9m2\" (UID: \"ff210be6-dbd3-451f-a26f-ad1c213d599d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 10:58:19.709829 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.709711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6z6h\" (UniqueName: \"kubernetes.io/projected/ff210be6-dbd3-451f-a26f-ad1c213d599d-kube-api-access-d6z6h\") pod \"kuadrant-operator-controller-manager-68cf589589-8z9m2\" (UID: \"ff210be6-dbd3-451f-a26f-ad1c213d599d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 10:58:19.710072 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.710053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff210be6-dbd3-451f-a26f-ad1c213d599d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8z9m2\" (UID: \"ff210be6-dbd3-451f-a26f-ad1c213d599d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 10:58:19.719146 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.719121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6z6h\" (UniqueName: \"kubernetes.io/projected/ff210be6-dbd3-451f-a26f-ad1c213d599d-kube-api-access-d6z6h\") pod \"kuadrant-operator-controller-manager-68cf589589-8z9m2\" (UID: \"ff210be6-dbd3-451f-a26f-ad1c213d599d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 10:58:19.822088 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.821996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 10:58:19.964109 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:19.964082    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"]
Aug 02 10:58:19.965303 ip-10-0-135-193 kubenswrapper[2576]: W0802 10:58:19.965276    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff210be6_dbd3_451f_a26f_ad1c213d599d.slice/crio-8e6ab718a675695154c07c415c75c5d888e6f908fe2f6a5a1bc8082947463315 WatchSource:0}: Error finding container 8e6ab718a675695154c07c415c75c5d888e6f908fe2f6a5a1bc8082947463315: Status 404 returned error can't find the container with id 8e6ab718a675695154c07c415c75c5d888e6f908fe2f6a5a1bc8082947463315
Aug 02 10:58:20.112705 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:20.112622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2" event={"ID":"ff210be6-dbd3-451f-a26f-ad1c213d599d","Type":"ContainerStarted","Data":"173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2"}
Aug 02 10:58:20.112705 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:20.112672    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2" event={"ID":"ff210be6-dbd3-451f-a26f-ad1c213d599d","Type":"ContainerStarted","Data":"8e6ab718a675695154c07c415c75c5d888e6f908fe2f6a5a1bc8082947463315"}
Aug 02 10:58:20.113107 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:20.112755    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 10:58:20.132564 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:20.132522    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2" podStartSLOduration=1.132505438 podStartE2EDuration="1.132505438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 10:58:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 10:58:20.131190979 +0000 UTC m=+1501.661773628" watchObservedRunningTime="2026-08-02 10:58:20.132505438 +0000 UTC m=+1501.663088066"
Aug 02 10:58:31.120808 ip-10-0-135-193 kubenswrapper[2576]: I0802 10:58:31.120773    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 11:03:18.993644 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:03:18.993534    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:03:19.001724 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:03:19.001700    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:06:14.729125 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.729094    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"]
Aug 02 11:06:14.732780 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.732759    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.735316 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.735298    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-07fc5de8-openshift-default-dockercfg-z8vv6\""
Aug 02 11:06:14.749288 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.749252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-certs\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.749397 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.749331    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-data\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.749459 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.749395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-socket\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.749518 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.749456    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istiod-ca-cert\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.749518 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.749493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c7pm\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-kube-api-access-5c7pm\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.749643 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.749519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-podinfo\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.749643 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.749591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-credential-socket\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.749754 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.749665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-token\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.749754 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.749695    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-envoy\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.752802 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.752779    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"]
Aug 02 11:06:14.850421 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850383    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-socket\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.850608 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850438    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istiod-ca-cert\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.850608 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c7pm\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-kube-api-access-5c7pm\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.850608 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-podinfo\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.850608 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-credential-socket\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.850821 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-token\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.850821 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-envoy\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.850821 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-certs\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.850821 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-data\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.851012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-socket\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.851012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.850899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-credential-socket\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.851171 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.851135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-data\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.851227 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.851206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-certs\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.851300 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.851282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istiod-ca-cert\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.853134 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.853108    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-envoy\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.853350 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.853334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-podinfo\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.857664 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.857638    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-token\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:14.857914 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:14.857898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c7pm\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-kube-api-access-5c7pm\") pod \"e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") " pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:15.043517 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:15.043433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:15.188381 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:15.188353    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"]
Aug 02 11:06:15.190280 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:06:15.190238    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb7b5e62_c774_4592_b1cc_d3f5e4e7ae4f.slice/crio-d997d90c1a82f4fb22007f109943e4d8fd57b019d849fcd522d96c753cd00bba WatchSource:0}: Error finding container d997d90c1a82f4fb22007f109943e4d8fd57b019d849fcd522d96c753cd00bba: Status 404 returned error can't find the container with id d997d90c1a82f4fb22007f109943e4d8fd57b019d849fcd522d96c753cd00bba
Aug 02 11:06:15.192748 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:15.192714    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:06:15.192850 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:15.192799    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:06:15.192899 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:15.192868    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:06:15.774981 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:15.774945    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" event={"ID":"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f","Type":"ContainerStarted","Data":"6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78"}
Aug 02 11:06:15.775380 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:15.775000    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" event={"ID":"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f","Type":"ContainerStarted","Data":"d997d90c1a82f4fb22007f109943e4d8fd57b019d849fcd522d96c753cd00bba"}
Aug 02 11:06:15.795615 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:15.795570    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" podStartSLOduration=1.795556222 podStartE2EDuration="1.795556222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:06:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:06:15.79328731 +0000 UTC m=+1977.323869938" watchObservedRunningTime="2026-08-02 11:06:15.795556222 +0000 UTC m=+1977.326138851"
Aug 02 11:06:16.043933 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:16.043847    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:16.048981 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:16.048956    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:16.781098 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:16.781057    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:16.782107 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:16.782091    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:18.279022 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.278991    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"]
Aug 02 11:06:18.282660 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.282639    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.285085 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.285061    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-07fc5de8-serving-cert\""
Aug 02 11:06:18.295479 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.295455    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"]
Aug 02 11:06:18.384453 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.384423    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0f1b004-5534-4e65-8950-14602d7cfd59-maas-api-tls\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.384646 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.384467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l66cr\" (UniqueName: \"kubernetes.io/projected/e0f1b004-5534-4e65-8950-14602d7cfd59-kube-api-access-l66cr\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.384646 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.384546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f1b004-5534-4e65-8950-14602d7cfd59-openshift-service-ca\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.485444 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.485403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0f1b004-5534-4e65-8950-14602d7cfd59-maas-api-tls\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.485444 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.485442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l66cr\" (UniqueName: \"kubernetes.io/projected/e0f1b004-5534-4e65-8950-14602d7cfd59-kube-api-access-l66cr\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.485708 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.485581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f1b004-5534-4e65-8950-14602d7cfd59-openshift-service-ca\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.485708 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:06:18.485582    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-serving-cert: secret "maas-api-e2e-ait-07fc5de8-serving-cert" not found
Aug 02 11:06:18.485817 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:06:18.485795    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0f1b004-5534-4e65-8950-14602d7cfd59-maas-api-tls podName:e0f1b004-5534-4e65-8950-14602d7cfd59 nodeName:}" failed. No retries permitted until 2026-08-02 11:06:18.985769832 +0000 UTC m=+1980.516352611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e0f1b004-5534-4e65-8950-14602d7cfd59-maas-api-tls") pod "maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz" (UID: "e0f1b004-5534-4e65-8950-14602d7cfd59") : secret "maas-api-e2e-ait-07fc5de8-serving-cert" not found
Aug 02 11:06:18.486296 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.486271    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f1b004-5534-4e65-8950-14602d7cfd59-openshift-service-ca\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.494473 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.494449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l66cr\" (UniqueName: \"kubernetes.io/projected/e0f1b004-5534-4e65-8950-14602d7cfd59-kube-api-access-l66cr\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.990806 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.990747    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0f1b004-5534-4e65-8950-14602d7cfd59-maas-api-tls\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:18.994301 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:18.994274    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0f1b004-5534-4e65-8950-14602d7cfd59-maas-api-tls\") pod \"maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:19.197139 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:19.197104    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:19.344911 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:19.344884    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"]
Aug 02 11:06:19.346755 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:06:19.346726    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f1b004_5534_4e65_8950_14602d7cfd59.slice/crio-c330ab102ee734813b71ecd7036208355b4e78f2c917b59a897a7c4cdecfc629 WatchSource:0}: Error finding container c330ab102ee734813b71ecd7036208355b4e78f2c917b59a897a7c4cdecfc629: Status 404 returned error can't find the container with id c330ab102ee734813b71ecd7036208355b4e78f2c917b59a897a7c4cdecfc629
Aug 02 11:06:19.348696 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:19.348672    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 02 11:06:19.799893 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:19.799862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz" event={"ID":"e0f1b004-5534-4e65-8950-14602d7cfd59","Type":"ContainerStarted","Data":"c330ab102ee734813b71ecd7036208355b4e78f2c917b59a897a7c4cdecfc629"}
Aug 02 11:06:20.807108 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:20.807073    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz" event={"ID":"e0f1b004-5534-4e65-8950-14602d7cfd59","Type":"ContainerStarted","Data":"e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079"}
Aug 02 11:06:20.807571 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:20.807213    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:20.824397 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:20.824348    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz" podStartSLOduration=2.231954061 podStartE2EDuration="2.824331546s" totalImagesPullingTime="592.377485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:06:20.822353007 +0000 UTC m=+1982.352935651" watchObservedRunningTime="2026-08-02 11:06:20.824331546 +0000 UTC m=+1982.354914175"
Aug 02 11:06:26.818703 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:26.818672    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:31.782650 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:31.782612    2576 patch_prober.go:28] interesting pod/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Aug 02 11:06:31.783098 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:31.782675    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Aug 02 11:06:31.874629 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:31.874593    2576 generic.go:356] "Generic (PLEG): container finished" podID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerID="6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78" exitCode=0
Aug 02 11:06:31.874800 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:31.874668    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" event={"ID":"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f","Type":"ContainerDied","Data":"6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78"}
Aug 02 11:06:31.875085 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:31.875068    2576 scope.go:117] "RemoveContainer" containerID="6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78"
Aug 02 11:06:31.876095 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:31.876067    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:06:31.876197 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:31.876148    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:06:31.876264 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:31.876199    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:06:32.882737 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:32.882702    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" event={"ID":"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f","Type":"ContainerStarted","Data":"8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088"}
Aug 02 11:06:33.043836 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:33.043801    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:33.044003 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:33.043977    2576 patch_prober.go:28] interesting pod/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Aug 02 11:06:33.044088 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:33.044023    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Aug 02 11:06:34.044232 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:34.044194    2576 patch_prober.go:28] interesting pod/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Aug 02 11:06:34.044629 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:34.044259    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Aug 02 11:06:35.043878 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:35.043835    2576 patch_prober.go:28] interesting pod/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Aug 02 11:06:35.044097 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:35.043898    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Aug 02 11:06:36.044287 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:36.044243    2576 patch_prober.go:28] interesting pod/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Aug 02 11:06:36.044768 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:36.044312    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Aug 02 11:06:37.044526 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:37.044490    2576 patch_prober.go:28] interesting pod/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Aug 02 11:06:37.044923 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:37.044549    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Aug 02 11:06:38.048577 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:38.048547    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:38.048969 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:38.048897    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:38.049538 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:38.049517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:42.929463 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:42.928771    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"]
Aug 02 11:06:42.929463 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:42.929001    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz" podUID="e0f1b004-5534-4e65-8950-14602d7cfd59" containerName="maas-api" containerID="cri-o://e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079" gracePeriod=30
Aug 02 11:06:43.198430 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.198393    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:43.238236 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.238199    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"]
Aug 02 11:06:43.238479 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.238443    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy" containerID="cri-o://8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088" gracePeriod=30
Aug 02 11:06:43.318734 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.318700    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l66cr\" (UniqueName: \"kubernetes.io/projected/e0f1b004-5534-4e65-8950-14602d7cfd59-kube-api-access-l66cr\") pod \"e0f1b004-5534-4e65-8950-14602d7cfd59\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") "
Aug 02 11:06:43.318734 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.318737    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f1b004-5534-4e65-8950-14602d7cfd59-openshift-service-ca\") pod \"e0f1b004-5534-4e65-8950-14602d7cfd59\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") "
Aug 02 11:06:43.318995 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.318826    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0f1b004-5534-4e65-8950-14602d7cfd59-maas-api-tls\") pod \"e0f1b004-5534-4e65-8950-14602d7cfd59\" (UID: \"e0f1b004-5534-4e65-8950-14602d7cfd59\") "
Aug 02 11:06:43.319220 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.319188    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0f1b004-5534-4e65-8950-14602d7cfd59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e0f1b004-5534-4e65-8950-14602d7cfd59" (UID: "e0f1b004-5534-4e65-8950-14602d7cfd59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:06:43.321127 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.321108    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f1b004-5534-4e65-8950-14602d7cfd59-kube-api-access-l66cr" (OuterVolumeSpecName: "kube-api-access-l66cr") pod "e0f1b004-5534-4e65-8950-14602d7cfd59" (UID: "e0f1b004-5534-4e65-8950-14602d7cfd59"). InnerVolumeSpecName "kube-api-access-l66cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:06:43.321215 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.321197    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0f1b004-5534-4e65-8950-14602d7cfd59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e0f1b004-5534-4e65-8950-14602d7cfd59" (UID: "e0f1b004-5534-4e65-8950-14602d7cfd59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:06:43.419783 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.419750    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0f1b004-5534-4e65-8950-14602d7cfd59-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:43.419783 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.419779    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l66cr\" (UniqueName: \"kubernetes.io/projected/e0f1b004-5534-4e65-8950-14602d7cfd59-kube-api-access-l66cr\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:43.419783 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.419790    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f1b004-5534-4e65-8950-14602d7cfd59-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:43.963911 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.963880    2576 generic.go:356] "Generic (PLEG): container finished" podID="e0f1b004-5534-4e65-8950-14602d7cfd59" containerID="e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079" exitCode=0
Aug 02 11:06:43.964392 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.963925    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"
Aug 02 11:06:43.964392 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.963972    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz" event={"ID":"e0f1b004-5534-4e65-8950-14602d7cfd59","Type":"ContainerDied","Data":"e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079"}
Aug 02 11:06:43.964392 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.964048    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz" event={"ID":"e0f1b004-5534-4e65-8950-14602d7cfd59","Type":"ContainerDied","Data":"c330ab102ee734813b71ecd7036208355b4e78f2c917b59a897a7c4cdecfc629"}
Aug 02 11:06:43.964392 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.964084    2576 scope.go:117] "RemoveContainer" containerID="e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079"
Aug 02 11:06:43.975419 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.975403    2576 scope.go:117] "RemoveContainer" containerID="e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079"
Aug 02 11:06:43.975649 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:06:43.975628    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079\": container with ID starting with e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079 not found: ID does not exist" containerID="e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079"
Aug 02 11:06:43.975695 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.975658    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079"} err="failed to get container status \"e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079\": rpc error: code = NotFound desc = could not find container \"e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079\": container with ID starting with e1dc03985c950c693fe1cac2229019cba3d130f6cbfabbb846738104249bb079 not found: ID does not exist"
Aug 02 11:06:43.986887 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.986866    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"]
Aug 02 11:06:43.992389 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:43.992370    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-07fc5de8-78fd55b6c4-86wvz"]
Aug 02 11:06:44.950071 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:44.950012    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0f1b004-5534-4e65-8950-14602d7cfd59" path="/var/lib/kubelet/pods/e0f1b004-5534-4e65-8950-14602d7cfd59/volumes"
Aug 02 11:06:46.646409 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.646373    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"]
Aug 02 11:06:46.647020 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.647005    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f1b004-5534-4e65-8950-14602d7cfd59" containerName="maas-api"
Aug 02 11:06:46.647102 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.647023    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f1b004-5534-4e65-8950-14602d7cfd59" containerName="maas-api"
Aug 02 11:06:46.647157 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.647146    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f1b004-5534-4e65-8950-14602d7cfd59" containerName="maas-api"
Aug 02 11:06:46.652269 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.652245    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:46.655282 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.655256    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-cc744106-serving-cert\""
Aug 02 11:06:46.659147 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.659119    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"]
Aug 02 11:06:46.753111 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.753077    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-openshift-service-ca\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:46.753283 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.753148    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb2wf\" (UniqueName: \"kubernetes.io/projected/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-kube-api-access-xb2wf\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:46.753283 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.753189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-maas-api-tls\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:46.853756 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.853721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-maas-api-tls\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:46.853938 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.853822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-openshift-service-ca\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:46.853938 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.853885    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb2wf\" (UniqueName: \"kubernetes.io/projected/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-kube-api-access-xb2wf\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:46.853938 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:06:46.853885    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-serving-cert: secret "maas-api-e2e-migrate-cc744106-serving-cert" not found
Aug 02 11:06:46.854100 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:06:46.854003    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-maas-api-tls podName:2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb nodeName:}" failed. No retries permitted until 2026-08-02 11:06:47.353980405 +0000 UTC m=+2008.884563018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-maas-api-tls") pod "maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz" (UID: "2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb") : secret "maas-api-e2e-migrate-cc744106-serving-cert" not found
Aug 02 11:06:46.854472 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.854454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-openshift-service-ca\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:46.864995 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:46.864970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb2wf\" (UniqueName: \"kubernetes.io/projected/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-kube-api-access-xb2wf\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:47.358570 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:47.358520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-maas-api-tls\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:47.361905 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:47.361878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-maas-api-tls\") pod \"maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:47.567771 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:47.567735    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:47.912215 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:47.912182    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"]
Aug 02 11:06:47.913524 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:06:47.913497    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc68c65_81ec_47b5_aa0c_fea6f5cce9eb.slice/crio-5dcd114cd984d4a08cc34edd3c8f5e43562233293df765bb953104370bd7b30c WatchSource:0}: Error finding container 5dcd114cd984d4a08cc34edd3c8f5e43562233293df765bb953104370bd7b30c: Status 404 returned error can't find the container with id 5dcd114cd984d4a08cc34edd3c8f5e43562233293df765bb953104370bd7b30c
Aug 02 11:06:47.987732 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:47.987690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz" event={"ID":"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb","Type":"ContainerStarted","Data":"5dcd114cd984d4a08cc34edd3c8f5e43562233293df765bb953104370bd7b30c"}
Aug 02 11:06:48.504121 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.504098    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:48.672224 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672193    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istiod-ca-cert\") pod \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") "
Aug 02 11:06:48.672403 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672237    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-credential-socket\") pod \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") "
Aug 02 11:06:48.672403 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672261    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-podinfo\") pod \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") "
Aug 02 11:06:48.672403 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672291    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-certs\") pod \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") "
Aug 02 11:06:48.672403 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672315    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-data\") pod \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") "
Aug 02 11:06:48.672403 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672329    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-envoy\") pod \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") "
Aug 02 11:06:48.672403 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672368    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c7pm\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-kube-api-access-5c7pm\") pod \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") "
Aug 02 11:06:48.672782 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672437    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-token\") pod \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") "
Aug 02 11:06:48.672782 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672469    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-socket\") pod \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\" (UID: \"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f\") "
Aug 02 11:06:48.672782 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672514    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" (UID: "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:06:48.672782 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672644    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" (UID: "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:06:48.672782 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672652    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" (UID: "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:06:48.673012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672811    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istiod-ca-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:48.673012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672826    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-data" (OuterVolumeSpecName: "istio-data") pod "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" (UID: "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:06:48.673012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672837    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" (UID: "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:06:48.673012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672832    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-credential-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:48.673012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.672872    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-certs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:48.674718 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.674689    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-kube-api-access-5c7pm" (OuterVolumeSpecName: "kube-api-access-5c7pm") pod "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" (UID: "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f"). InnerVolumeSpecName "kube-api-access-5c7pm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:06:48.674827 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.674785    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" (UID: "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 02 11:06:48.675169 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.675146    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" (UID: "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:06:48.675238 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.675223    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-token" (OuterVolumeSpecName: "istio-token") pod "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" (UID: "db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:06:48.774160 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.774119    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-podinfo\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:48.774160 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.774156    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-data\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:48.774329 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.774168    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-envoy\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:48.774329 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.774180    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c7pm\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-kube-api-access-5c7pm\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:48.774329 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.774194    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-istio-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:48.774329 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.774207    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f-workload-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:06:48.996205 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.996177    2576 generic.go:356] "Generic (PLEG): container finished" podID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerID="8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088" exitCode=0
Aug 02 11:06:48.996617 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.996237    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"
Aug 02 11:06:48.996617 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.996263    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" event={"ID":"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f","Type":"ContainerDied","Data":"8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088"}
Aug 02 11:06:48.996617 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.996313    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt" event={"ID":"db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f","Type":"ContainerDied","Data":"d997d90c1a82f4fb22007f109943e4d8fd57b019d849fcd522d96c753cd00bba"}
Aug 02 11:06:48.996617 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.996348    2576 scope.go:117] "RemoveContainer" containerID="8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088"
Aug 02 11:06:48.998735 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.998198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz" event={"ID":"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb","Type":"ContainerStarted","Data":"1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523"}
Aug 02 11:06:48.998735 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:48.998417    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:06:49.006591 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:49.006573    2576 scope.go:117] "RemoveContainer" containerID="6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78"
Aug 02 11:06:49.015155 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:49.015137    2576 scope.go:117] "RemoveContainer" containerID="8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088"
Aug 02 11:06:49.015225 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:49.015161    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"]
Aug 02 11:06:49.015398 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:06:49.015371    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088\": container with ID starting with 8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088 not found: ID does not exist" containerID="8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088"
Aug 02 11:06:49.015450 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:49.015397    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088"} err="failed to get container status \"8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088\": rpc error: code = NotFound desc = could not find container \"8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088\": container with ID starting with 8ff7cc92dc2a448cc6091b76d3094b3cd88018a775d238ac844e130282081088 not found: ID does not exist"
Aug 02 11:06:49.015450 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:49.015413    2576 scope.go:117] "RemoveContainer" containerID="6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78"
Aug 02 11:06:49.015618 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:06:49.015599    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78\": container with ID starting with 6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78 not found: ID does not exist" containerID="6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78"
Aug 02 11:06:49.015694 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:49.015629    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78"} err="failed to get container status \"6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78\": rpc error: code = NotFound desc = could not find container \"6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78\": container with ID starting with 6a951fc40d97fe048af33f28668ba6eb4189cf69729e39d696758c90a9c73c78 not found: ID does not exist"
Aug 02 11:06:49.020138 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:49.020120    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-07fc5de8-openshift-default-7598c5f6f6-2z9xt"]
Aug 02 11:06:49.033612 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:49.033574    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz" podStartSLOduration=2.584604134 podStartE2EDuration="3.033563075s" totalImagesPullingTime="448.958941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:06:49.031937157 +0000 UTC m=+2010.562519816" watchObservedRunningTime="2026-08-02 11:06:49.033563075 +0000 UTC m=+2010.564145708"
Aug 02 11:06:50.950632 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:50.950591    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" path="/var/lib/kubelet/pods/db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f/volumes"
Aug 02 11:06:55.011225 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:06:55.011195    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:07:11.609776 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:11.609737    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"]
Aug 02 11:07:11.610313 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:11.609996    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz" podUID="2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb" containerName="maas-api" containerID="cri-o://1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523" gracePeriod=30
Aug 02 11:07:11.898962 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:11.898927    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:07:11.975875 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:11.975842    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-openshift-service-ca\") pod \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") "
Aug 02 11:07:11.975875 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:11.975882    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-maas-api-tls\") pod \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") "
Aug 02 11:07:11.976165 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:11.975973    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb2wf\" (UniqueName: \"kubernetes.io/projected/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-kube-api-access-xb2wf\") pod \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\" (UID: \"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb\") "
Aug 02 11:07:11.976360 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:11.976329    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb" (UID: "2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:07:11.978306 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:11.978270    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb" (UID: "2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:07:11.978424 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:11.978302    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-kube-api-access-xb2wf" (OuterVolumeSpecName: "kube-api-access-xb2wf") pod "2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb" (UID: "2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb"). InnerVolumeSpecName "kube-api-access-xb2wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:07:12.077477 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.077435    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb2wf\" (UniqueName: \"kubernetes.io/projected/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-kube-api-access-xb2wf\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:12.077477 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.077473    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:12.077477 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.077484    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:12.129506 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.129401    2576 generic.go:356] "Generic (PLEG): container finished" podID="2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb" containerID="1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523" exitCode=0
Aug 02 11:07:12.129506 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.129446    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"
Aug 02 11:07:12.129506 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.129493    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz" event={"ID":"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb","Type":"ContainerDied","Data":"1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523"}
Aug 02 11:07:12.129780 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.129552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz" event={"ID":"2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb","Type":"ContainerDied","Data":"5dcd114cd984d4a08cc34edd3c8f5e43562233293df765bb953104370bd7b30c"}
Aug 02 11:07:12.129780 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.129577    2576 scope.go:117] "RemoveContainer" containerID="1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523"
Aug 02 11:07:12.139482 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.139462    2576 scope.go:117] "RemoveContainer" containerID="1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523"
Aug 02 11:07:12.139734 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:07:12.139713    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523\": container with ID starting with 1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523 not found: ID does not exist" containerID="1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523"
Aug 02 11:07:12.139790 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.139745    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523"} err="failed to get container status \"1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523\": rpc error: code = NotFound desc = could not find container \"1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523\": container with ID starting with 1b6d1556281bdb80f925a4a1b0537e4d5baef7b5cc6ad2204990515211007523 not found: ID does not exist"
Aug 02 11:07:12.170794 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.170763    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"]
Aug 02 11:07:12.184932 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.184906    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cc744106-b845c8cd5-wckrz"]
Aug 02 11:07:12.625393 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.625362    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"]
Aug 02 11:07:12.625978 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.625957    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy"
Aug 02 11:07:12.626141 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.625981    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy"
Aug 02 11:07:12.626141 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.625999    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy"
Aug 02 11:07:12.626141 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.626007    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy"
Aug 02 11:07:12.626141 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.626021    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb" containerName="maas-api"
Aug 02 11:07:12.626141 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.626049    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb" containerName="maas-api"
Aug 02 11:07:12.626403 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.626165    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy"
Aug 02 11:07:12.626403 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.626182    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb" containerName="maas-api"
Aug 02 11:07:12.630846 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.630821    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.633561 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.633536    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a9cb70b3-openshift-default-dockercfg-tc4td\""
Aug 02 11:07:12.642567 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.642540    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"]
Aug 02 11:07:12.683541 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.683514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547a26ca-747f-4844-b828-39746c960992-istio-podinfo\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.683722 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.683565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmmzn\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-kube-api-access-kmmzn\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.683722 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.683633    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-istio-token\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.683722 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.683698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-socket\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.683862 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.683728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-certs\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.683862 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.683765    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-credential-socket\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.683862 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.683782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-envoy\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.683862 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.683811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-data\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.683862 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.683844    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547a26ca-747f-4844-b828-39746c960992-istiod-ca-cert\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.785728 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.785688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmmzn\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-kube-api-access-kmmzn\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.785926 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.785777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-istio-token\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.785926 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.785837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-socket\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.785926 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.785874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-certs\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.785926 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.785919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-credential-socket\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.786172 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.785948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-envoy\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.786172 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.785983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-data\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.786172 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.786013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547a26ca-747f-4844-b828-39746c960992-istiod-ca-cert\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.786172 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.786088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547a26ca-747f-4844-b828-39746c960992-istio-podinfo\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.786420 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.786393    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-certs\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.787108 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.786996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-socket\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.787460 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.787435    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-data\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.787703 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.787682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-credential-socket\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.788308 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.788278    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547a26ca-747f-4844-b828-39746c960992-istiod-ca-cert\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.796222 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.796197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547a26ca-747f-4844-b828-39746c960992-istio-podinfo\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.798276 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.798246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-envoy\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.799515 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.799489    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmmzn\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-kube-api-access-kmmzn\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.800096 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.800067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-istio-token\") pod \"e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") " pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.944761 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.944723    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:12.951437 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:12.951414    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb" path="/var/lib/kubelet/pods/2fc68c65-81ec-47b5-aa0c-fea6f5cce9eb/volumes"
Aug 02 11:07:13.088766 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:13.088734    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"]
Aug 02 11:07:13.090412 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:07:13.090385    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547a26ca_747f_4844_b828_39746c960992.slice/crio-b10d2ef349e895ab85e89367c57902290216238dd0d22f47d1cc2325939f16dd WatchSource:0}: Error finding container b10d2ef349e895ab85e89367c57902290216238dd0d22f47d1cc2325939f16dd: Status 404 returned error can't find the container with id b10d2ef349e895ab85e89367c57902290216238dd0d22f47d1cc2325939f16dd
Aug 02 11:07:13.092778 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:13.092742    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:07:13.092868 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:13.092826    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:07:13.092932 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:13.092868    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:07:13.137550 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:13.137522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62" event={"ID":"547a26ca-747f-4844-b828-39746c960992","Type":"ContainerStarted","Data":"b10d2ef349e895ab85e89367c57902290216238dd0d22f47d1cc2325939f16dd"}
Aug 02 11:07:14.146172 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:14.146135    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62" event={"ID":"547a26ca-747f-4844-b828-39746c960992","Type":"ContainerStarted","Data":"cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596"}
Aug 02 11:07:14.168244 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:14.168189    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62" podStartSLOduration=2.168173117 podStartE2EDuration="2.168173117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:07:14.167173305 +0000 UTC m=+2035.697755954" watchObservedRunningTime="2026-08-02 11:07:14.168173117 +0000 UTC m=+2035.698755745"
Aug 02 11:07:14.950893 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:14.950867    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:14.951097 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:14.950963    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:15.151752 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:15.151719    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:15.152862 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:15.152844    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:16.561609 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.561569    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"]
Aug 02 11:07:16.562102 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.562077    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="db7b5e62-c774-4592-b1cc-d3f5e4e7ae4f" containerName="istio-proxy"
Aug 02 11:07:16.565234 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.565211    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.567883 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.567862    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a9cb70b3-serving-cert\""
Aug 02 11:07:16.576973 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.576944    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"]
Aug 02 11:07:16.622555 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.622513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcnfb\" (UniqueName: \"kubernetes.io/projected/b2645f06-a331-40c2-adea-e607b9c006ef-kube-api-access-kcnfb\") pod \"maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.622752 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.622562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2645f06-a331-40c2-adea-e607b9c006ef-maas-api-tls\") pod \"maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.622752 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.622676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2645f06-a331-40c2-adea-e607b9c006ef-openshift-service-ca\") pod \"maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.723639 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.723604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcnfb\" (UniqueName: \"kubernetes.io/projected/b2645f06-a331-40c2-adea-e607b9c006ef-kube-api-access-kcnfb\") pod \"maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.723876 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.723648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2645f06-a331-40c2-adea-e607b9c006ef-maas-api-tls\") pod \"maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.723876 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.723705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2645f06-a331-40c2-adea-e607b9c006ef-openshift-service-ca\") pod \"maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.724373 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.724340    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2645f06-a331-40c2-adea-e607b9c006ef-openshift-service-ca\") pod \"maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.726370 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.726344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2645f06-a331-40c2-adea-e607b9c006ef-maas-api-tls\") pod \"maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.731472 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.731446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcnfb\" (UniqueName: \"kubernetes.io/projected/b2645f06-a331-40c2-adea-e607b9c006ef-kube-api-access-kcnfb\") pod \"maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:16.878738 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:16.878640    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:17.034056 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:17.033996    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"]
Aug 02 11:07:17.036875 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:07:17.036421    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2645f06_a331_40c2_adea_e607b9c006ef.slice/crio-c7d0d88050238d334769540199a3d6052b4f9442b4074b82db8a9cd2246a1c14 WatchSource:0}: Error finding container c7d0d88050238d334769540199a3d6052b4f9442b4074b82db8a9cd2246a1c14: Status 404 returned error can't find the container with id c7d0d88050238d334769540199a3d6052b4f9442b4074b82db8a9cd2246a1c14
Aug 02 11:07:17.164049 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:17.164002    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp" event={"ID":"b2645f06-a331-40c2-adea-e607b9c006ef","Type":"ContainerStarted","Data":"c7d0d88050238d334769540199a3d6052b4f9442b4074b82db8a9cd2246a1c14"}
Aug 02 11:07:18.172594 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:18.172553    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp" event={"ID":"b2645f06-a331-40c2-adea-e607b9c006ef","Type":"ContainerStarted","Data":"d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915"}
Aug 02 11:07:18.172996 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:18.172662    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:18.189498 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:18.189433    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp" podStartSLOduration=1.682298686 podStartE2EDuration="2.189412534s" totalImagesPullingTime="507.113848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:07:18.188422001 +0000 UTC m=+2039.719004630" watchObservedRunningTime="2026-08-02 11:07:18.189412534 +0000 UTC m=+2039.719995164"
Aug 02 11:07:24.183714 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:24.183674    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:41.496946 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.496337    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"]
Aug 02 11:07:41.496946 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.496577    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp" podUID="b2645f06-a331-40c2-adea-e607b9c006ef" containerName="maas-api" containerID="cri-o://d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915" gracePeriod=30
Aug 02 11:07:41.767334 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.767313    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:41.860963 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.860935    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcnfb\" (UniqueName: \"kubernetes.io/projected/b2645f06-a331-40c2-adea-e607b9c006ef-kube-api-access-kcnfb\") pod \"b2645f06-a331-40c2-adea-e607b9c006ef\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") "
Aug 02 11:07:41.861148 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.860992    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2645f06-a331-40c2-adea-e607b9c006ef-openshift-service-ca\") pod \"b2645f06-a331-40c2-adea-e607b9c006ef\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") "
Aug 02 11:07:41.861148 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.861055    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2645f06-a331-40c2-adea-e607b9c006ef-maas-api-tls\") pod \"b2645f06-a331-40c2-adea-e607b9c006ef\" (UID: \"b2645f06-a331-40c2-adea-e607b9c006ef\") "
Aug 02 11:07:41.861377 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.861352    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2645f06-a331-40c2-adea-e607b9c006ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b2645f06-a331-40c2-adea-e607b9c006ef" (UID: "b2645f06-a331-40c2-adea-e607b9c006ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:07:41.863235 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.863209    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2645f06-a331-40c2-adea-e607b9c006ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b2645f06-a331-40c2-adea-e607b9c006ef" (UID: "b2645f06-a331-40c2-adea-e607b9c006ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:07:41.863365 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.863257    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2645f06-a331-40c2-adea-e607b9c006ef-kube-api-access-kcnfb" (OuterVolumeSpecName: "kube-api-access-kcnfb") pod "b2645f06-a331-40c2-adea-e607b9c006ef" (UID: "b2645f06-a331-40c2-adea-e607b9c006ef"). InnerVolumeSpecName "kube-api-access-kcnfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:07:41.961959 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.961935    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcnfb\" (UniqueName: \"kubernetes.io/projected/b2645f06-a331-40c2-adea-e607b9c006ef-kube-api-access-kcnfb\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:41.961959 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.961958    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2645f06-a331-40c2-adea-e607b9c006ef-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:41.962176 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:41.961968    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2645f06-a331-40c2-adea-e607b9c006ef-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:42.312807 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.312757    2576 generic.go:356] "Generic (PLEG): container finished" podID="b2645f06-a331-40c2-adea-e607b9c006ef" containerID="d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915" exitCode=0
Aug 02 11:07:42.313044 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.312880    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp" event={"ID":"b2645f06-a331-40c2-adea-e607b9c006ef","Type":"ContainerDied","Data":"d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915"}
Aug 02 11:07:42.313044 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.312931    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp" event={"ID":"b2645f06-a331-40c2-adea-e607b9c006ef","Type":"ContainerDied","Data":"c7d0d88050238d334769540199a3d6052b4f9442b4074b82db8a9cd2246a1c14"}
Aug 02 11:07:42.313044 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.312960    2576 scope.go:117] "RemoveContainer" containerID="d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915"
Aug 02 11:07:42.313190 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.313162    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"
Aug 02 11:07:42.326505 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.326486    2576 scope.go:117] "RemoveContainer" containerID="d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915"
Aug 02 11:07:42.326766 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:07:42.326749    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915\": container with ID starting with d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915 not found: ID does not exist" containerID="d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915"
Aug 02 11:07:42.326813 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.326775    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915"} err="failed to get container status \"d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915\": rpc error: code = NotFound desc = could not find container \"d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915\": container with ID starting with d92c0372a1e465d8fb0d1909eafc2d424770ff040064bb0bd4909a91d9b36915 not found: ID does not exist"
Aug 02 11:07:42.339020 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.338998    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"]
Aug 02 11:07:42.341322 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.341303    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9cb70b3-6457bd5b8f-kjbsp"]
Aug 02 11:07:42.750517 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.750486    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"]
Aug 02 11:07:42.750898 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.750734    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62" podUID="547a26ca-747f-4844-b828-39746c960992" containerName="istio-proxy" containerID="cri-o://cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596" gracePeriod=30
Aug 02 11:07:42.957512 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:42.957474    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2645f06-a331-40c2-adea-e607b9c006ef" path="/var/lib/kubelet/pods/b2645f06-a331-40c2-adea-e607b9c006ef/volumes"
Aug 02 11:07:45.187985 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:45.187950    2576 patch_prober.go:28] interesting pod/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 02 11:07:45.188566 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:45.188538    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62" podUID="547a26ca-747f-4844-b828-39746c960992" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 02 11:07:46.953820 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:46.953781    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"]
Aug 02 11:07:46.954765 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:46.954514    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2645f06-a331-40c2-adea-e607b9c006ef" containerName="maas-api"
Aug 02 11:07:46.954765 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:46.954533    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2645f06-a331-40c2-adea-e607b9c006ef" containerName="maas-api"
Aug 02 11:07:46.954765 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:46.954636    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2645f06-a331-40c2-adea-e607b9c006ef" containerName="maas-api"
Aug 02 11:07:46.959445 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:46.959423    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:46.962232 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:46.962210    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-75b9be26-serving-cert\""
Aug 02 11:07:46.968533 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:46.968511    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"]
Aug 02 11:07:47.114765 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.114723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-maas-api-tls\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.115382 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.115359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-openshift-service-ca\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.115631 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.115615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lwnb\" (UniqueName: \"kubernetes.io/projected/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-kube-api-access-4lwnb\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.218490 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.216772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-maas-api-tls\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.218490 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.216837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-openshift-service-ca\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.218490 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.216911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lwnb\" (UniqueName: \"kubernetes.io/projected/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-kube-api-access-4lwnb\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.218490 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:07:47.217375    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-serving-cert: secret "maas-api-e2e-derive-75b9be26-serving-cert" not found
Aug 02 11:07:47.218490 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:07:47.217437    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-maas-api-tls podName:fbbb9aa4-1a9c-45c0-928d-c651c292ce42 nodeName:}" failed. No retries permitted until 2026-08-02 11:07:47.717417879 +0000 UTC m=+2069.248000509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-maas-api-tls") pod "maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq" (UID: "fbbb9aa4-1a9c-45c0-928d-c651c292ce42") : secret "maas-api-e2e-derive-75b9be26-serving-cert" not found
Aug 02 11:07:47.218490 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.218391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-openshift-service-ca\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.229600 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.229572    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lwnb\" (UniqueName: \"kubernetes.io/projected/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-kube-api-access-4lwnb\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.722142 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.722111    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-maas-api-tls\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.724670 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.724637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-maas-api-tls\") pod \"maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:47.873067 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:47.872961    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:48.026495 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:48.026461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"]
Aug 02 11:07:48.028991 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:07:48.028960    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbb9aa4_1a9c_45c0_928d_c651c292ce42.slice/crio-8296f4ac07b40557b7a6ad08b9a2824f5d4d16f14eebbaffaa20a7d290c18903 WatchSource:0}: Error finding container 8296f4ac07b40557b7a6ad08b9a2824f5d4d16f14eebbaffaa20a7d290c18903: Status 404 returned error can't find the container with id 8296f4ac07b40557b7a6ad08b9a2824f5d4d16f14eebbaffaa20a7d290c18903
Aug 02 11:07:48.362718 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:48.362681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq" event={"ID":"fbbb9aa4-1a9c-45c0-928d-c651c292ce42","Type":"ContainerStarted","Data":"8296f4ac07b40557b7a6ad08b9a2824f5d4d16f14eebbaffaa20a7d290c18903"}
Aug 02 11:07:49.370486 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:49.370437    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq" event={"ID":"fbbb9aa4-1a9c-45c0-928d-c651c292ce42","Type":"ContainerStarted","Data":"7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d"}
Aug 02 11:07:49.370951 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:49.370533    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:07:49.390376 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:49.390322    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq" podStartSLOduration=3.061005513 podStartE2EDuration="3.390307712s" totalImagesPullingTime="329.302199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:07:49.387374401 +0000 UTC m=+2070.917957045" watchObservedRunningTime="2026-08-02 11:07:49.390307712 +0000 UTC m=+2070.920890406"
Aug 02 11:07:50.728555 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.728531    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:50.852679 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852585    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547a26ca-747f-4844-b828-39746c960992-istio-podinfo\") pod \"547a26ca-747f-4844-b828-39746c960992\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") "
Aug 02 11:07:50.852679 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852620    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-data\") pod \"547a26ca-747f-4844-b828-39746c960992\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") "
Aug 02 11:07:50.852679 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852635    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-envoy\") pod \"547a26ca-747f-4844-b828-39746c960992\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") "
Aug 02 11:07:50.852679 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852655    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmmzn\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-kube-api-access-kmmzn\") pod \"547a26ca-747f-4844-b828-39746c960992\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") "
Aug 02 11:07:50.853019 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852691    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-socket\") pod \"547a26ca-747f-4844-b828-39746c960992\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") "
Aug 02 11:07:50.853019 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-certs\") pod \"547a26ca-747f-4844-b828-39746c960992\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") "
Aug 02 11:07:50.853019 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852744    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-istio-token\") pod \"547a26ca-747f-4844-b828-39746c960992\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") "
Aug 02 11:07:50.853019 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852767    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-credential-socket\") pod \"547a26ca-747f-4844-b828-39746c960992\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") "
Aug 02 11:07:50.853019 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852825    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547a26ca-747f-4844-b828-39746c960992-istiod-ca-cert\") pod \"547a26ca-747f-4844-b828-39746c960992\" (UID: \"547a26ca-747f-4844-b828-39746c960992\") "
Aug 02 11:07:50.853019 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.852944    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-data" (OuterVolumeSpecName: "istio-data") pod "547a26ca-747f-4844-b828-39746c960992" (UID: "547a26ca-747f-4844-b828-39746c960992"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:07:50.853370 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.853113    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "547a26ca-747f-4844-b828-39746c960992" (UID: "547a26ca-747f-4844-b828-39746c960992"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:07:50.853370 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.853158    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "547a26ca-747f-4844-b828-39746c960992" (UID: "547a26ca-747f-4844-b828-39746c960992"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:07:50.853370 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.853252    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-credential-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:50.853370 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.853272    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-data\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:50.853370 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.853285    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-certs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:50.853370 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.853318    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "547a26ca-747f-4844-b828-39746c960992" (UID: "547a26ca-747f-4844-b828-39746c960992"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:07:50.853370 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.853346    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/547a26ca-747f-4844-b828-39746c960992-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "547a26ca-747f-4844-b828-39746c960992" (UID: "547a26ca-747f-4844-b828-39746c960992"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:07:50.855120 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.855094    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/547a26ca-747f-4844-b828-39746c960992-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "547a26ca-747f-4844-b828-39746c960992" (UID: "547a26ca-747f-4844-b828-39746c960992"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 02 11:07:50.855120 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.855106    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "547a26ca-747f-4844-b828-39746c960992" (UID: "547a26ca-747f-4844-b828-39746c960992"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:07:50.855285 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.855180    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-kube-api-access-kmmzn" (OuterVolumeSpecName: "kube-api-access-kmmzn") pod "547a26ca-747f-4844-b828-39746c960992" (UID: "547a26ca-747f-4844-b828-39746c960992"). InnerVolumeSpecName "kube-api-access-kmmzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:07:50.855285 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.855274    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-istio-token" (OuterVolumeSpecName: "istio-token") pod "547a26ca-747f-4844-b828-39746c960992" (UID: "547a26ca-747f-4844-b828-39746c960992"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:07:50.953837 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.953797    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-workload-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:50.953837 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.953832    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-istio-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:50.954069 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.953847    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/547a26ca-747f-4844-b828-39746c960992-istiod-ca-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:50.954069 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.953861    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/547a26ca-747f-4844-b828-39746c960992-istio-podinfo\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:50.954069 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.953874    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/547a26ca-747f-4844-b828-39746c960992-istio-envoy\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:50.954069 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:50.953888    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmmzn\" (UniqueName: \"kubernetes.io/projected/547a26ca-747f-4844-b828-39746c960992-kube-api-access-kmmzn\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:07:51.383253 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:51.383222    2576 generic.go:356] "Generic (PLEG): container finished" podID="547a26ca-747f-4844-b828-39746c960992" containerID="cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596" exitCode=0
Aug 02 11:07:51.383430 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:51.383264    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"
Aug 02 11:07:51.383430 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:51.383302    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62" event={"ID":"547a26ca-747f-4844-b828-39746c960992","Type":"ContainerDied","Data":"cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596"}
Aug 02 11:07:51.383430 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:51.383356    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62" event={"ID":"547a26ca-747f-4844-b828-39746c960992","Type":"ContainerDied","Data":"b10d2ef349e895ab85e89367c57902290216238dd0d22f47d1cc2325939f16dd"}
Aug 02 11:07:51.383430 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:51.383379    2576 scope.go:117] "RemoveContainer" containerID="cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596"
Aug 02 11:07:51.394307 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:51.394286    2576 scope.go:117] "RemoveContainer" containerID="cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596"
Aug 02 11:07:51.394598 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:07:51.394577    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596\": container with ID starting with cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596 not found: ID does not exist" containerID="cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596"
Aug 02 11:07:51.394692 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:51.394610    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596"} err="failed to get container status \"cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596\": rpc error: code = NotFound desc = could not find container \"cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596\": container with ID starting with cfde2e7bb06254ea1cae364f1bc1d038d08560f48313579f6a027a1e6d930596 not found: ID does not exist"
Aug 02 11:07:51.402407 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:51.402386    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"]
Aug 02 11:07:51.410840 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:51.410817    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a9cb70b3-openshift-default-5f44bf9bb-bvw62"]
Aug 02 11:07:52.951256 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:52.951224    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="547a26ca-747f-4844-b828-39746c960992" path="/var/lib/kubelet/pods/547a26ca-747f-4844-b828-39746c960992/volumes"
Aug 02 11:07:55.382450 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:07:55.382415    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:08:10.730269 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:10.730234    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"]
Aug 02 11:08:10.730743 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:10.730493    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq" podUID="fbbb9aa4-1a9c-45c0-928d-c651c292ce42" containerName="maas-api" containerID="cri-o://7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d" gracePeriod=30
Aug 02 11:08:11.002831 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.002796    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:08:11.124788 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.124753    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-maas-api-tls\") pod \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") "
Aug 02 11:08:11.124968 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.124801    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lwnb\" (UniqueName: \"kubernetes.io/projected/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-kube-api-access-4lwnb\") pod \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") "
Aug 02 11:08:11.124968 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.124856    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-openshift-service-ca\") pod \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\" (UID: \"fbbb9aa4-1a9c-45c0-928d-c651c292ce42\") "
Aug 02 11:08:11.125289 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.125266    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fbbb9aa4-1a9c-45c0-928d-c651c292ce42" (UID: "fbbb9aa4-1a9c-45c0-928d-c651c292ce42"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:08:11.126943 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.126919    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fbbb9aa4-1a9c-45c0-928d-c651c292ce42" (UID: "fbbb9aa4-1a9c-45c0-928d-c651c292ce42"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:08:11.127067 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.127017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-kube-api-access-4lwnb" (OuterVolumeSpecName: "kube-api-access-4lwnb") pod "fbbb9aa4-1a9c-45c0-928d-c651c292ce42" (UID: "fbbb9aa4-1a9c-45c0-928d-c651c292ce42"). InnerVolumeSpecName "kube-api-access-4lwnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:08:11.226017 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.225974    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lwnb\" (UniqueName: \"kubernetes.io/projected/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-kube-api-access-4lwnb\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:08:11.226017 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.226012    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:08:11.226228 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.226054    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbbb9aa4-1a9c-45c0-928d-c651c292ce42-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:08:11.500340 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.500310    2576 generic.go:356] "Generic (PLEG): container finished" podID="fbbb9aa4-1a9c-45c0-928d-c651c292ce42" containerID="7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d" exitCode=0
Aug 02 11:08:11.500544 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.500377    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"
Aug 02 11:08:11.500544 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.500400    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq" event={"ID":"fbbb9aa4-1a9c-45c0-928d-c651c292ce42","Type":"ContainerDied","Data":"7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d"}
Aug 02 11:08:11.500544 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.500479    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq" event={"ID":"fbbb9aa4-1a9c-45c0-928d-c651c292ce42","Type":"ContainerDied","Data":"8296f4ac07b40557b7a6ad08b9a2824f5d4d16f14eebbaffaa20a7d290c18903"}
Aug 02 11:08:11.500544 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.500505    2576 scope.go:117] "RemoveContainer" containerID="7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d"
Aug 02 11:08:11.510807 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.510792    2576 scope.go:117] "RemoveContainer" containerID="7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d"
Aug 02 11:08:11.511093 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:08:11.511074    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d\": container with ID starting with 7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d not found: ID does not exist" containerID="7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d"
Aug 02 11:08:11.511163 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.511105    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d"} err="failed to get container status \"7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d\": rpc error: code = NotFound desc = could not find container \"7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d\": container with ID starting with 7b03be334eb360f46695dd4e7df35023ff5b22d70b7066624a8af0d8a24a761d not found: ID does not exist"
Aug 02 11:08:11.523582 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.523562    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"]
Aug 02 11:08:11.527100 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:11.527071    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75b9be26-55bf4dfcb9-98qrq"]
Aug 02 11:08:12.950817 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:12.950784    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbbb9aa4-1a9c-45c0-928d-c651c292ce42" path="/var/lib/kubelet/pods/fbbb9aa4-1a9c-45c0-928d-c651c292ce42/volumes"
Aug 02 11:08:19.030296 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:19.030261    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:08:19.041534 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:08:19.041511    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:10:08.787003 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.786973    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"]
Aug 02 11:10:08.787602 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.787581    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="547a26ca-747f-4844-b828-39746c960992" containerName="istio-proxy"
Aug 02 11:10:08.787666 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.787606    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="547a26ca-747f-4844-b828-39746c960992" containerName="istio-proxy"
Aug 02 11:10:08.787666 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.787641    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbbb9aa4-1a9c-45c0-928d-c651c292ce42" containerName="maas-api"
Aug 02 11:10:08.787666 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.787649    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbbb9aa4-1a9c-45c0-928d-c651c292ce42" containerName="maas-api"
Aug 02 11:10:08.787773 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.787751    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="547a26ca-747f-4844-b828-39746c960992" containerName="istio-proxy"
Aug 02 11:10:08.787773 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.787764    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbbb9aa4-1a9c-45c0-928d-c651c292ce42" containerName="maas-api"
Aug 02 11:10:08.791717 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.791695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.794409 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.794385    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8126169a-openshift-default-dockercfg-xcmm4\""
Aug 02 11:10:08.811573 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.811548    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"]
Aug 02 11:10:08.850145 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.850103    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-token\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.850301 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.850151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-podinfo\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.850301 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.850268    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-certs\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.850391 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.850299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-766qw\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-kube-api-access-766qw\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.850391 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.850341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-socket\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.850391 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.850368    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istiod-ca-cert\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.850497 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.850403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-data\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.850497 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.850430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-envoy\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.850497 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.850458    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-credential-socket\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.951855 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.951818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-certs\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952047 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.951868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-766qw\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-kube-api-access-766qw\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952047 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.951896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-socket\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952047 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.951924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istiod-ca-cert\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952047 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.951948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-data\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952047 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.951980    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-envoy\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952047 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.952006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-credential-socket\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952368 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.952102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-token\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952368 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.952130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-podinfo\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952368 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.952285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-certs\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.952368 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.952356    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-socket\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.953069 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.952625    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-data\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.953203 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.953096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-credential-socket\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.953269 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.953239    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istiod-ca-cert\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.954595 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.954568    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-envoy\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.954827 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.954806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-podinfo\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.960720 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.960692    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-766qw\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-kube-api-access-766qw\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:08.960817 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:08.960696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-token\") pod \"e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") " pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:09.103985 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:09.103891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:09.255964 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:09.255936    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"]
Aug 02 11:10:09.257250 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:10:09.257226    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae95d51c_8612_4f50_a93e_6ac2ba982be2.slice/crio-1cd5370fb0bad1e3d158f0173d16b06f27fa5b105b646e2f0ed994b2151f35aa WatchSource:0}: Error finding container 1cd5370fb0bad1e3d158f0173d16b06f27fa5b105b646e2f0ed994b2151f35aa: Status 404 returned error can't find the container with id 1cd5370fb0bad1e3d158f0173d16b06f27fa5b105b646e2f0ed994b2151f35aa
Aug 02 11:10:09.259259 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:09.259220    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:10:09.259338 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:09.259312    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:10:09.259376 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:09.259360    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:10:10.176310 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:10.176275    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" event={"ID":"ae95d51c-8612-4f50-a93e-6ac2ba982be2","Type":"ContainerStarted","Data":"b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53"}
Aug 02 11:10:10.176671 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:10.176326    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" event={"ID":"ae95d51c-8612-4f50-a93e-6ac2ba982be2","Type":"ContainerStarted","Data":"1cd5370fb0bad1e3d158f0173d16b06f27fa5b105b646e2f0ed994b2151f35aa"}
Aug 02 11:10:10.197687 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:10.197641    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" podStartSLOduration=2.197626698 podStartE2EDuration="2.197626698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:10:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:10:10.195965853 +0000 UTC m=+2211.726548514" watchObservedRunningTime="2026-08-02 11:10:10.197626698 +0000 UTC m=+2211.728209327"
Aug 02 11:10:11.104871 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:11.104836    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:11.110572 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:11.110546    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:11.181971 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:11.181939    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:11.182902 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:11.182881    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:13.249914 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.249885    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"]
Aug 02 11:10:13.253707 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.253680    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.256016 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.255995    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8126169a-serving-cert\""
Aug 02 11:10:13.266748 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.264640    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"]
Aug 02 11:10:13.388992 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.388961    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjg4h\" (UniqueName: \"kubernetes.io/projected/2077199f-4bd2-44c7-8cc3-81c009b60791-kube-api-access-tjg4h\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.389185 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.389056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2077199f-4bd2-44c7-8cc3-81c009b60791-openshift-service-ca\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.389185 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.389115    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2077199f-4bd2-44c7-8cc3-81c009b60791-maas-api-tls\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.489824 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.489786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2077199f-4bd2-44c7-8cc3-81c009b60791-maas-api-tls\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.489994 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.489844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjg4h\" (UniqueName: \"kubernetes.io/projected/2077199f-4bd2-44c7-8cc3-81c009b60791-kube-api-access-tjg4h\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.489994 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.489932    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2077199f-4bd2-44c7-8cc3-81c009b60791-openshift-service-ca\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.489994 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:10:13.489948    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-serving-cert: secret "maas-api-e2e-mt-8126169a-serving-cert" not found
Aug 02 11:10:13.490176 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:10:13.490067    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2077199f-4bd2-44c7-8cc3-81c009b60791-maas-api-tls podName:2077199f-4bd2-44c7-8cc3-81c009b60791 nodeName:}" failed. No retries permitted until 2026-08-02 11:10:13.99001601 +0000 UTC m=+2215.520598618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2077199f-4bd2-44c7-8cc3-81c009b60791-maas-api-tls") pod "maas-api-e2e-mt-8126169a-759f4c8965-fnc8p" (UID: "2077199f-4bd2-44c7-8cc3-81c009b60791") : secret "maas-api-e2e-mt-8126169a-serving-cert" not found
Aug 02 11:10:13.490721 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.490697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2077199f-4bd2-44c7-8cc3-81c009b60791-openshift-service-ca\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.498644 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.498609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjg4h\" (UniqueName: \"kubernetes.io/projected/2077199f-4bd2-44c7-8cc3-81c009b60791-kube-api-access-tjg4h\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.993686 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.993650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2077199f-4bd2-44c7-8cc3-81c009b60791-maas-api-tls\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:13.996578 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:13.996537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2077199f-4bd2-44c7-8cc3-81c009b60791-maas-api-tls\") pod \"maas-api-e2e-mt-8126169a-759f4c8965-fnc8p\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:14.170504 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:14.170465    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:14.316799 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:14.316773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"]
Aug 02 11:10:14.318773 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:10:14.318748    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2077199f_4bd2_44c7_8cc3_81c009b60791.slice/crio-a0e72e4a0f6709870e78db92ebf1130a5c02769c10d0702770c719358417c127 WatchSource:0}: Error finding container a0e72e4a0f6709870e78db92ebf1130a5c02769c10d0702770c719358417c127: Status 404 returned error can't find the container with id a0e72e4a0f6709870e78db92ebf1130a5c02769c10d0702770c719358417c127
Aug 02 11:10:15.206762 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:15.206718    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p" event={"ID":"2077199f-4bd2-44c7-8cc3-81c009b60791","Type":"ContainerStarted","Data":"0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc"}
Aug 02 11:10:15.206949 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:15.206794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p" event={"ID":"2077199f-4bd2-44c7-8cc3-81c009b60791","Type":"ContainerStarted","Data":"a0e72e4a0f6709870e78db92ebf1130a5c02769c10d0702770c719358417c127"}
Aug 02 11:10:15.206949 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:15.206912    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:15.224437 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:15.224388    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p" podStartSLOduration=1.825362217 podStartE2EDuration="2.224371566s" totalImagesPullingTime="399.009349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:10:15.221838362 +0000 UTC m=+2216.752420991" watchObservedRunningTime="2026-08-02 11:10:15.224371566 +0000 UTC m=+2216.754954195"
Aug 02 11:10:21.219744 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:21.219712    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:26.184201 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:26.184167    2576 patch_prober.go:28] interesting pod/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Aug 02 11:10:26.184648 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:26.184230    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Aug 02 11:10:26.272479 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:26.272448    2576 generic.go:356] "Generic (PLEG): container finished" podID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerID="b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53" exitCode=0
Aug 02 11:10:26.272650 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:26.272530    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" event={"ID":"ae95d51c-8612-4f50-a93e-6ac2ba982be2","Type":"ContainerDied","Data":"b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53"}
Aug 02 11:10:26.272934 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:26.272921    2576 scope.go:117] "RemoveContainer" containerID="b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53"
Aug 02 11:10:26.273922 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:26.273894    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:10:26.273998 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:26.273981    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:10:26.274067 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:26.274053    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:10:27.280174 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:27.280138    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" event={"ID":"ae95d51c-8612-4f50-a93e-6ac2ba982be2","Type":"ContainerStarted","Data":"5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9"}
Aug 02 11:10:28.105053 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:28.104994    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:28.105570 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:28.105543    2576 patch_prober.go:28] interesting pod/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Aug 02 11:10:28.105799 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:28.105762    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Aug 02 11:10:29.104516 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:29.104466    2576 patch_prober.go:28] interesting pod/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Aug 02 11:10:29.105019 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:29.104541    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Aug 02 11:10:30.104852 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:30.104811    2576 patch_prober.go:28] interesting pod/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Aug 02 11:10:30.105481 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:30.105450    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Aug 02 11:10:31.104717 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:31.104644    2576 patch_prober.go:28] interesting pod/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Aug 02 11:10:31.104934 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:31.104741    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Aug 02 11:10:32.104578 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:32.104540    2576 patch_prober.go:28] interesting pod/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Aug 02 11:10:32.104858 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:32.104830    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Aug 02 11:10:33.108142 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:33.108114    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:33.108566 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:33.108428    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:33.109016 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:33.108998    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:44.520827 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.520640    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"]
Aug 02 11:10:44.521344 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.521309    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p" podUID="2077199f-4bd2-44c7-8cc3-81c009b60791" containerName="maas-api" containerID="cri-o://0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc" gracePeriod=30
Aug 02 11:10:44.793989 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.793959    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:44.869803 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.869760    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2077199f-4bd2-44c7-8cc3-81c009b60791-openshift-service-ca\") pod \"2077199f-4bd2-44c7-8cc3-81c009b60791\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") "
Aug 02 11:10:44.870021 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.869833    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2077199f-4bd2-44c7-8cc3-81c009b60791-maas-api-tls\") pod \"2077199f-4bd2-44c7-8cc3-81c009b60791\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") "
Aug 02 11:10:44.870021 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.869913    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjg4h\" (UniqueName: \"kubernetes.io/projected/2077199f-4bd2-44c7-8cc3-81c009b60791-kube-api-access-tjg4h\") pod \"2077199f-4bd2-44c7-8cc3-81c009b60791\" (UID: \"2077199f-4bd2-44c7-8cc3-81c009b60791\") "
Aug 02 11:10:44.870563 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.870529    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2077199f-4bd2-44c7-8cc3-81c009b60791-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2077199f-4bd2-44c7-8cc3-81c009b60791" (UID: "2077199f-4bd2-44c7-8cc3-81c009b60791"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:10:44.872372 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.872343    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2077199f-4bd2-44c7-8cc3-81c009b60791-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2077199f-4bd2-44c7-8cc3-81c009b60791" (UID: "2077199f-4bd2-44c7-8cc3-81c009b60791"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:10:44.872906 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.872863    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2077199f-4bd2-44c7-8cc3-81c009b60791-kube-api-access-tjg4h" (OuterVolumeSpecName: "kube-api-access-tjg4h") pod "2077199f-4bd2-44c7-8cc3-81c009b60791" (UID: "2077199f-4bd2-44c7-8cc3-81c009b60791"). InnerVolumeSpecName "kube-api-access-tjg4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:10:44.970864 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.970836    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjg4h\" (UniqueName: \"kubernetes.io/projected/2077199f-4bd2-44c7-8cc3-81c009b60791-kube-api-access-tjg4h\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:44.970864 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.970862    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2077199f-4bd2-44c7-8cc3-81c009b60791-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:44.971017 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:44.970872    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2077199f-4bd2-44c7-8cc3-81c009b60791-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:45.391459 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:45.391426    2576 generic.go:356] "Generic (PLEG): container finished" podID="2077199f-4bd2-44c7-8cc3-81c009b60791" containerID="0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc" exitCode=0
Aug 02 11:10:45.391653 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:45.391488    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"
Aug 02 11:10:45.391653 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:45.391514    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p" event={"ID":"2077199f-4bd2-44c7-8cc3-81c009b60791","Type":"ContainerDied","Data":"0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc"}
Aug 02 11:10:45.391653 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:45.391566    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p" event={"ID":"2077199f-4bd2-44c7-8cc3-81c009b60791","Type":"ContainerDied","Data":"a0e72e4a0f6709870e78db92ebf1130a5c02769c10d0702770c719358417c127"}
Aug 02 11:10:45.391653 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:45.391591    2576 scope.go:117] "RemoveContainer" containerID="0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc"
Aug 02 11:10:45.401089 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:45.401014    2576 scope.go:117] "RemoveContainer" containerID="0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc"
Aug 02 11:10:45.401294 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:10:45.401271    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc\": container with ID starting with 0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc not found: ID does not exist" containerID="0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc"
Aug 02 11:10:45.401361 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:45.401306    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc"} err="failed to get container status \"0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc\": rpc error: code = NotFound desc = could not find container \"0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc\": container with ID starting with 0d08ac5a2b051a362bd7e3d9cd82d908ac3e28959cb81e240df93acabac955dc not found: ID does not exist"
Aug 02 11:10:45.408929 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:45.408905    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"]
Aug 02 11:10:45.413710 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:45.413683    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8126169a-759f4c8965-fnc8p"]
Aug 02 11:10:46.951559 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:46.951530    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2077199f-4bd2-44c7-8cc3-81c009b60791" path="/var/lib/kubelet/pods/2077199f-4bd2-44c7-8cc3-81c009b60791/volumes"
Aug 02 11:10:51.342883 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:51.342850    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"]
Aug 02 11:10:51.343406 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:51.343078    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy" containerID="cri-o://5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9" gracePeriod=30
Aug 02 11:10:56.599732 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.599701    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:56.669182 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669157    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-envoy\") pod \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") "
Aug 02 11:10:56.669334 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669195    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-certs\") pod \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") "
Aug 02 11:10:56.669334 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669216    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-token\") pod \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") "
Aug 02 11:10:56.669334 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669243    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istiod-ca-cert\") pod \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") "
Aug 02 11:10:56.669334 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669322    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-data\") pod \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") "
Aug 02 11:10:56.669546 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669441    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-socket\") pod \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") "
Aug 02 11:10:56.669546 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669453    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae95d51c-8612-4f50-a93e-6ac2ba982be2" (UID: "ae95d51c-8612-4f50-a93e-6ac2ba982be2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:10:56.669546 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669500    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-credential-socket\") pod \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") "
Aug 02 11:10:56.669546 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669543    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-podinfo\") pod \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") "
Aug 02 11:10:56.669750 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669578    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-766qw\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-kube-api-access-766qw\") pod \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\" (UID: \"ae95d51c-8612-4f50-a93e-6ac2ba982be2\") "
Aug 02 11:10:56.669750 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669610    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae95d51c-8612-4f50-a93e-6ac2ba982be2" (UID: "ae95d51c-8612-4f50-a93e-6ac2ba982be2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:10:56.669750 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669715    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae95d51c-8612-4f50-a93e-6ac2ba982be2" (UID: "ae95d51c-8612-4f50-a93e-6ac2ba982be2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:10:56.669912 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669768    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae95d51c-8612-4f50-a93e-6ac2ba982be2" (UID: "ae95d51c-8612-4f50-a93e-6ac2ba982be2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:10:56.669912 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669798    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae95d51c-8612-4f50-a93e-6ac2ba982be2" (UID: "ae95d51c-8612-4f50-a93e-6ac2ba982be2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:10:56.670082 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.669960    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-data\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:56.670162 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.670090    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:56.670162 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.670114    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-credential-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:56.670162 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.670128    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-workload-certs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:56.670162 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.670142    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istiod-ca-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:56.671531 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.671512    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae95d51c-8612-4f50-a93e-6ac2ba982be2" (UID: "ae95d51c-8612-4f50-a93e-6ac2ba982be2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:10:56.671633 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.671534    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae95d51c-8612-4f50-a93e-6ac2ba982be2" (UID: "ae95d51c-8612-4f50-a93e-6ac2ba982be2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:10:56.672176 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.672159    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae95d51c-8612-4f50-a93e-6ac2ba982be2" (UID: "ae95d51c-8612-4f50-a93e-6ac2ba982be2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 02 11:10:56.672176 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.672169    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-kube-api-access-766qw" (OuterVolumeSpecName: "kube-api-access-766qw") pod "ae95d51c-8612-4f50-a93e-6ac2ba982be2" (UID: "ae95d51c-8612-4f50-a93e-6ac2ba982be2"). InnerVolumeSpecName "kube-api-access-766qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:10:56.770813 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.770784    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-podinfo\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:56.770813 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.770811    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-766qw\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-kube-api-access-766qw\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:56.770959 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.770825    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-envoy\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:56.770959 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:56.770837    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae95d51c-8612-4f50-a93e-6ac2ba982be2-istio-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:10:57.461460 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.461420    2576 generic.go:356] "Generic (PLEG): container finished" podID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerID="5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9" exitCode=0
Aug 02 11:10:57.461626 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.461476    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"
Aug 02 11:10:57.461626 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.461503    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" event={"ID":"ae95d51c-8612-4f50-a93e-6ac2ba982be2","Type":"ContainerDied","Data":"5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9"}
Aug 02 11:10:57.461626 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.461561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk" event={"ID":"ae95d51c-8612-4f50-a93e-6ac2ba982be2","Type":"ContainerDied","Data":"1cd5370fb0bad1e3d158f0173d16b06f27fa5b105b646e2f0ed994b2151f35aa"}
Aug 02 11:10:57.461626 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.461584    2576 scope.go:117] "RemoveContainer" containerID="5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9"
Aug 02 11:10:57.470948 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.470933    2576 scope.go:117] "RemoveContainer" containerID="b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53"
Aug 02 11:10:57.479890 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.479874    2576 scope.go:117] "RemoveContainer" containerID="5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9"
Aug 02 11:10:57.480209 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:10:57.480190    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9\": container with ID starting with 5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9 not found: ID does not exist" containerID="5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9"
Aug 02 11:10:57.480286 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.480217    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9"} err="failed to get container status \"5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9\": rpc error: code = NotFound desc = could not find container \"5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9\": container with ID starting with 5de33e5f74e4506f3ad94521120d8919a29ffd95f6430a541e9ac427c069f1c9 not found: ID does not exist"
Aug 02 11:10:57.480286 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.480233    2576 scope.go:117] "RemoveContainer" containerID="b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53"
Aug 02 11:10:57.480497 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:10:57.480470    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53\": container with ID starting with b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53 not found: ID does not exist" containerID="b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53"
Aug 02 11:10:57.480543 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.480502    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53"} err="failed to get container status \"b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53\": rpc error: code = NotFound desc = could not find container \"b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53\": container with ID starting with b694c6c37f25e2ae13b312932c72014a9678780dc162c51991f989664570ed53 not found: ID does not exist"
Aug 02 11:10:57.480941 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.480923    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"]
Aug 02 11:10:57.484793 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:57.484773    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8126169a-openshift-default-7dcb7d5ffb-72xgk"]
Aug 02 11:10:58.950954 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:10:58.950918    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" path="/var/lib/kubelet/pods/ae95d51c-8612-4f50-a93e-6ac2ba982be2/volumes"
Aug 02 11:11:32.253689 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.253653    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"]
Aug 02 11:11:32.254223 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.253839    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2" podUID="ff210be6-dbd3-451f-a26f-ad1c213d599d" containerName="manager" containerID="cri-o://173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2" gracePeriod=10
Aug 02 11:11:32.520719 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.520693    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 11:11:32.660383 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.660347    2576 generic.go:356] "Generic (PLEG): container finished" podID="ff210be6-dbd3-451f-a26f-ad1c213d599d" containerID="173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2" exitCode=0
Aug 02 11:11:32.660564 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.660389    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"
Aug 02 11:11:32.660564 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.660433    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2" event={"ID":"ff210be6-dbd3-451f-a26f-ad1c213d599d","Type":"ContainerDied","Data":"173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2"}
Aug 02 11:11:32.660564 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.660487    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2" event={"ID":"ff210be6-dbd3-451f-a26f-ad1c213d599d","Type":"ContainerDied","Data":"8e6ab718a675695154c07c415c75c5d888e6f908fe2f6a5a1bc8082947463315"}
Aug 02 11:11:32.660564 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.660514    2576 scope.go:117] "RemoveContainer" containerID="173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2"
Aug 02 11:11:32.670917 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.670897    2576 scope.go:117] "RemoveContainer" containerID="173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2"
Aug 02 11:11:32.671230 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:11:32.671209    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2\": container with ID starting with 173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2 not found: ID does not exist" containerID="173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2"
Aug 02 11:11:32.671314 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.671240    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2"} err="failed to get container status \"173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2\": rpc error: code = NotFound desc = could not find container \"173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2\": container with ID starting with 173389a2cd29771778098df8869f24d88b84a5f77a09accbb8a0797fd6fb62a2 not found: ID does not exist"
Aug 02 11:11:32.687661 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.687639    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6z6h\" (UniqueName: \"kubernetes.io/projected/ff210be6-dbd3-451f-a26f-ad1c213d599d-kube-api-access-d6z6h\") pod \"ff210be6-dbd3-451f-a26f-ad1c213d599d\" (UID: \"ff210be6-dbd3-451f-a26f-ad1c213d599d\") "
Aug 02 11:11:32.687750 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.687683    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff210be6-dbd3-451f-a26f-ad1c213d599d-extensions-socket-volume\") pod \"ff210be6-dbd3-451f-a26f-ad1c213d599d\" (UID: \"ff210be6-dbd3-451f-a26f-ad1c213d599d\") "
Aug 02 11:11:32.688073 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.688023    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff210be6-dbd3-451f-a26f-ad1c213d599d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ff210be6-dbd3-451f-a26f-ad1c213d599d" (UID: "ff210be6-dbd3-451f-a26f-ad1c213d599d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:11:32.689778 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.689753    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff210be6-dbd3-451f-a26f-ad1c213d599d-kube-api-access-d6z6h" (OuterVolumeSpecName: "kube-api-access-d6z6h") pod "ff210be6-dbd3-451f-a26f-ad1c213d599d" (UID: "ff210be6-dbd3-451f-a26f-ad1c213d599d"). InnerVolumeSpecName "kube-api-access-d6z6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:11:32.788491 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.788418    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff210be6-dbd3-451f-a26f-ad1c213d599d-extensions-socket-volume\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:11:32.788491 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.788443    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6z6h\" (UniqueName: \"kubernetes.io/projected/ff210be6-dbd3-451f-a26f-ad1c213d599d-kube-api-access-d6z6h\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:11:32.976864 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.976833    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"]
Aug 02 11:11:32.982235 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:32.982213    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8z9m2"]
Aug 02 11:11:34.950813 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:11:34.950779    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff210be6-dbd3-451f-a26f-ad1c213d599d" path="/var/lib/kubelet/pods/ff210be6-dbd3-451f-a26f-ad1c213d599d/volumes"
Aug 02 11:12:47.861065 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861018    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"]
Aug 02 11:12:47.861521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861420    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff210be6-dbd3-451f-a26f-ad1c213d599d" containerName="manager"
Aug 02 11:12:47.861521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861432    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff210be6-dbd3-451f-a26f-ad1c213d599d" containerName="manager"
Aug 02 11:12:47.861521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861455    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy"
Aug 02 11:12:47.861521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861461    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy"
Aug 02 11:12:47.861521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861469    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy"
Aug 02 11:12:47.861521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861475    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy"
Aug 02 11:12:47.861521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861484    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2077199f-4bd2-44c7-8cc3-81c009b60791" containerName="maas-api"
Aug 02 11:12:47.861521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861489    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2077199f-4bd2-44c7-8cc3-81c009b60791" containerName="maas-api"
Aug 02 11:12:47.861771 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861542    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy"
Aug 02 11:12:47.861771 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861551    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff210be6-dbd3-451f-a26f-ad1c213d599d" containerName="manager"
Aug 02 11:12:47.861771 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861558    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae95d51c-8612-4f50-a93e-6ac2ba982be2" containerName="istio-proxy"
Aug 02 11:12:47.861771 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.861565    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2077199f-4bd2-44c7-8cc3-81c009b60791" containerName="maas-api"
Aug 02 11:12:47.864661 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.864645    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:12:47.867307 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.867284    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fjhc6\""
Aug 02 11:12:47.876414 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.876392    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"]
Aug 02 11:12:47.938211 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.938171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v97fr\" (UniqueName: \"kubernetes.io/projected/6a2dc4ef-5878-419a-8138-1bb069901ef0-kube-api-access-v97fr\") pod \"kuadrant-operator-controller-manager-68cf589589-w26zf\" (UID: \"6a2dc4ef-5878-419a-8138-1bb069901ef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:12:47.938426 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:47.938351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a2dc4ef-5878-419a-8138-1bb069901ef0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-w26zf\" (UID: \"6a2dc4ef-5878-419a-8138-1bb069901ef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:12:48.039700 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:48.039640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a2dc4ef-5878-419a-8138-1bb069901ef0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-w26zf\" (UID: \"6a2dc4ef-5878-419a-8138-1bb069901ef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:12:48.039886 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:48.039715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v97fr\" (UniqueName: \"kubernetes.io/projected/6a2dc4ef-5878-419a-8138-1bb069901ef0-kube-api-access-v97fr\") pod \"kuadrant-operator-controller-manager-68cf589589-w26zf\" (UID: \"6a2dc4ef-5878-419a-8138-1bb069901ef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:12:48.040229 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:48.040206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a2dc4ef-5878-419a-8138-1bb069901ef0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-w26zf\" (UID: \"6a2dc4ef-5878-419a-8138-1bb069901ef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:12:48.051136 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:48.051108    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v97fr\" (UniqueName: \"kubernetes.io/projected/6a2dc4ef-5878-419a-8138-1bb069901ef0-kube-api-access-v97fr\") pod \"kuadrant-operator-controller-manager-68cf589589-w26zf\" (UID: \"6a2dc4ef-5878-419a-8138-1bb069901ef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:12:48.176188 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:48.176155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:12:48.320604 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:48.320579    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"]
Aug 02 11:12:48.323642 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:12:48.323618    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2dc4ef_5878_419a_8138_1bb069901ef0.slice/crio-29282ac36fb901928389fec1c8f49bd1cbae14b4579af14a971179112f244d35 WatchSource:0}: Error finding container 29282ac36fb901928389fec1c8f49bd1cbae14b4579af14a971179112f244d35: Status 404 returned error can't find the container with id 29282ac36fb901928389fec1c8f49bd1cbae14b4579af14a971179112f244d35
Aug 02 11:12:49.073880 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:49.073845    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf" event={"ID":"6a2dc4ef-5878-419a-8138-1bb069901ef0","Type":"ContainerStarted","Data":"780678bff16adfb24316b43581bec3b6ea1a7f907911e489d6e911e4ccbd171d"}
Aug 02 11:12:49.074288 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:49.073897    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf" event={"ID":"6a2dc4ef-5878-419a-8138-1bb069901ef0","Type":"ContainerStarted","Data":"29282ac36fb901928389fec1c8f49bd1cbae14b4579af14a971179112f244d35"}
Aug 02 11:12:49.074288 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:49.073981    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:12:49.123462 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:12:49.123414    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf" podStartSLOduration=2.12339742 podStartE2EDuration="2.12339742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:12:49.120005803 +0000 UTC m=+2370.650588449" watchObservedRunningTime="2026-08-02 11:12:49.12339742 +0000 UTC m=+2370.653980049"
Aug 02 11:13:00.081536 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:00.081503    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w26zf"
Aug 02 11:13:06.801158 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.801120    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"]
Aug 02 11:13:06.805405 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.805386    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:06.808229 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.808208    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a9dd0457-openshift-default-dockercfg-cppm6\""
Aug 02 11:13:06.816842 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.816757    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"]
Aug 02 11:13:06.908598 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.908551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-certs\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:06.908598 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.908598    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-envoy\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:06.908845 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.908617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-podinfo\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:06.908845 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.908732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26dq7\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-kube-api-access-26dq7\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:06.908845 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.908769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-token\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:06.909013 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.908853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-credential-socket\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:06.909013 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.908896    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-socket\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:06.909013 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.908992    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istiod-ca-cert\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:06.909211 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:06.909048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-data\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010146 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-certs\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010336 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010167    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-envoy\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010336 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-podinfo\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010476 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26dq7\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-kube-api-access-26dq7\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010476 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-token\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010476 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-credential-socket\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010666 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010490    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-socket\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010666 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-certs\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010666 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010641    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istiod-ca-cert\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010857 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-data\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010857 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-credential-socket\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010981 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-socket\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.010981 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.010966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-data\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.011322 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.011246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istiod-ca-cert\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.012615 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.012594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-envoy\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.013166 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.013147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-podinfo\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.021583 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.021559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-token\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.025965 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.025943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26dq7\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-kube-api-access-26dq7\") pod \"e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") " pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.117967 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.117885    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:07.267744 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.267717    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"]
Aug 02 11:13:07.269112 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:13:07.269085    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda64b08fa_fcea_4bda_99d2_d0ca5eb581b8.slice/crio-bf972fc0b1093ff65a67ed04669130b5c5af8104f403739573621a619c6a7311 WatchSource:0}: Error finding container bf972fc0b1093ff65a67ed04669130b5c5af8104f403739573621a619c6a7311: Status 404 returned error can't find the container with id bf972fc0b1093ff65a67ed04669130b5c5af8104f403739573621a619c6a7311
Aug 02 11:13:07.271198 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.271165    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:13:07.271270 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.271227    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:13:07.271270 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:07.271259    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:13:08.183238 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:08.183199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" event={"ID":"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8","Type":"ContainerStarted","Data":"b698903559d5810ba0705e54e9c5e10f5bab718bb41cb66b7c8e085d27a11119"}
Aug 02 11:13:08.183624 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:08.183267    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" event={"ID":"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8","Type":"ContainerStarted","Data":"bf972fc0b1093ff65a67ed04669130b5c5af8104f403739573621a619c6a7311"}
Aug 02 11:13:08.203430 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:08.203382    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" podStartSLOduration=2.203367558 podStartE2EDuration="2.203367558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:13:08.202710451 +0000 UTC m=+2389.733293081" watchObservedRunningTime="2026-08-02 11:13:08.203367558 +0000 UTC m=+2389.733950186"
Aug 02 11:13:09.118483 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:09.118450    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:09.123341 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:09.123320    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:09.188970 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:09.188942    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:09.189980 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:09.189960    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:11.193757 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.193720    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"]
Aug 02 11:13:11.198126 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.198104    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.200484 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.200467    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a9dd0457-serving-cert\""
Aug 02 11:13:11.207466 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.207444    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"]
Aug 02 11:13:11.353579 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.353544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f81322ae-0b32-434d-867a-f972c95a282c-maas-api-tls\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.353579 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.353575    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdnn4\" (UniqueName: \"kubernetes.io/projected/f81322ae-0b32-434d-867a-f972c95a282c-kube-api-access-xdnn4\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.353848 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.353735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81322ae-0b32-434d-867a-f972c95a282c-openshift-service-ca\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.455078 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.454973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81322ae-0b32-434d-867a-f972c95a282c-openshift-service-ca\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.455222 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.455098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f81322ae-0b32-434d-867a-f972c95a282c-maas-api-tls\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.455222 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.455125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdnn4\" (UniqueName: \"kubernetes.io/projected/f81322ae-0b32-434d-867a-f972c95a282c-kube-api-access-xdnn4\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.455312 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:13:11.455234    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-serving-cert: secret "maas-api-e2e-mt-a9dd0457-serving-cert" not found
Aug 02 11:13:11.455312 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:13:11.455305    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f81322ae-0b32-434d-867a-f972c95a282c-maas-api-tls podName:f81322ae-0b32-434d-867a-f972c95a282c nodeName:}" failed. No retries permitted until 2026-08-02 11:13:11.955289405 +0000 UTC m=+2393.485872015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f81322ae-0b32-434d-867a-f972c95a282c-maas-api-tls") pod "maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl" (UID: "f81322ae-0b32-434d-867a-f972c95a282c") : secret "maas-api-e2e-mt-a9dd0457-serving-cert" not found
Aug 02 11:13:11.455647 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.455629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81322ae-0b32-434d-867a-f972c95a282c-openshift-service-ca\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.463517 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.463491    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdnn4\" (UniqueName: \"kubernetes.io/projected/f81322ae-0b32-434d-867a-f972c95a282c-kube-api-access-xdnn4\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.959931 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.959900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f81322ae-0b32-434d-867a-f972c95a282c-maas-api-tls\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:11.962666 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:11.962641    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f81322ae-0b32-434d-867a-f972c95a282c-maas-api-tls\") pod \"maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:12.114002 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:12.113965    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:12.265177 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:12.265152    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"]
Aug 02 11:13:12.267004 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:13:12.266965    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf81322ae_0b32_434d_867a_f972c95a282c.slice/crio-332ca37073a08feb634599abdb71a906392ecb87d3e31e88b4fd22e82a6ce60a WatchSource:0}: Error finding container 332ca37073a08feb634599abdb71a906392ecb87d3e31e88b4fd22e82a6ce60a: Status 404 returned error can't find the container with id 332ca37073a08feb634599abdb71a906392ecb87d3e31e88b4fd22e82a6ce60a
Aug 02 11:13:12.268414 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:12.268396    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 02 11:13:13.216196 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:13.216164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl" event={"ID":"f81322ae-0b32-434d-867a-f972c95a282c","Type":"ContainerStarted","Data":"b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42"}
Aug 02 11:13:13.216393 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:13.216214    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl" event={"ID":"f81322ae-0b32-434d-867a-f972c95a282c","Type":"ContainerStarted","Data":"332ca37073a08feb634599abdb71a906392ecb87d3e31e88b4fd22e82a6ce60a"}
Aug 02 11:13:13.216393 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:13.216258    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:13.233406 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:13.233368    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl" podStartSLOduration=1.681934458 podStartE2EDuration="2.233354496s" totalImagesPullingTime="551.420038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:13:13.231541061 +0000 UTC m=+2394.762123718" watchObservedRunningTime="2026-08-02 11:13:13.233354496 +0000 UTC m=+2394.763937125"
Aug 02 11:13:19.070344 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:19.070320    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:13:19.083007 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:19.082989    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:13:19.228219 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:19.228186    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:13:24.945635 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:24.945599    2576 patch_prober.go:28] interesting pod/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": read tcp 10.133.0.2:56858->10.133.0.67:15021: read: connection reset by peer" start-of-body=
Aug 02 11:13:24.946057 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:24.945684    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": read tcp 10.133.0.2:56858->10.133.0.67:15021: read: connection reset by peer"
Aug 02 11:13:25.285758 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:25.285670    2576 generic.go:356] "Generic (PLEG): container finished" podID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerID="b698903559d5810ba0705e54e9c5e10f5bab718bb41cb66b7c8e085d27a11119" exitCode=0
Aug 02 11:13:25.285758 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:25.285719    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" event={"ID":"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8","Type":"ContainerDied","Data":"b698903559d5810ba0705e54e9c5e10f5bab718bb41cb66b7c8e085d27a11119"}
Aug 02 11:13:25.286213 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:25.286182    2576 scope.go:117] "RemoveContainer" containerID="b698903559d5810ba0705e54e9c5e10f5bab718bb41cb66b7c8e085d27a11119"
Aug 02 11:13:25.287280 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:25.287251    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:13:25.287370 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:25.287336    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:13:25.287433 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:25.287398    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:13:26.296153 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:26.296107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" event={"ID":"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8","Type":"ContainerStarted","Data":"56be0e97723b403e5f2d0815a841072dfa0a8c059d3617d882fe1998304e5018"}
Aug 02 11:13:27.118779 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:27.118742    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:27.118987 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:27.118961    2576 patch_prober.go:28] interesting pod/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Aug 02 11:13:27.119081 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:27.119022    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Aug 02 11:13:28.118846 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:28.118804    2576 patch_prober.go:28] interesting pod/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Aug 02 11:13:28.119333 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:28.118874    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Aug 02 11:13:29.118767 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:29.118724    2576 patch_prober.go:28] interesting pod/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Aug 02 11:13:29.118960 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:29.118785    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Aug 02 11:13:30.118660 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:30.118619    2576 patch_prober.go:28] interesting pod/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Aug 02 11:13:30.118844 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:30.118677    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Aug 02 11:13:31.118827 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:31.118786    2576 patch_prober.go:28] interesting pod/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Aug 02 11:13:31.119266 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:31.118845    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Aug 02 11:13:32.122334 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:32.122300    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:32.122714 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:32.122662    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:13:32.123449 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:13:32.123427    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:17:27.814054 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:27.813979    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"]
Aug 02 11:17:27.814687 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:27.814291    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl" podUID="f81322ae-0b32-434d-867a-f972c95a282c" containerName="maas-api" containerID="cri-o://b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42" gracePeriod=30
Aug 02 11:17:28.072016 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.071951    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:17:28.073901 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.073880    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdnn4\" (UniqueName: \"kubernetes.io/projected/f81322ae-0b32-434d-867a-f972c95a282c-kube-api-access-xdnn4\") pod \"f81322ae-0b32-434d-867a-f972c95a282c\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") "
Aug 02 11:17:28.074022 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.073918    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81322ae-0b32-434d-867a-f972c95a282c-openshift-service-ca\") pod \"f81322ae-0b32-434d-867a-f972c95a282c\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") "
Aug 02 11:17:28.074022 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.073981    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f81322ae-0b32-434d-867a-f972c95a282c-maas-api-tls\") pod \"f81322ae-0b32-434d-867a-f972c95a282c\" (UID: \"f81322ae-0b32-434d-867a-f972c95a282c\") "
Aug 02 11:17:28.074329 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.074295    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f81322ae-0b32-434d-867a-f972c95a282c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f81322ae-0b32-434d-867a-f972c95a282c" (UID: "f81322ae-0b32-434d-867a-f972c95a282c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:17:28.076300 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.076277    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f81322ae-0b32-434d-867a-f972c95a282c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f81322ae-0b32-434d-867a-f972c95a282c" (UID: "f81322ae-0b32-434d-867a-f972c95a282c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:17:28.076411 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.076334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f81322ae-0b32-434d-867a-f972c95a282c-kube-api-access-xdnn4" (OuterVolumeSpecName: "kube-api-access-xdnn4") pod "f81322ae-0b32-434d-867a-f972c95a282c" (UID: "f81322ae-0b32-434d-867a-f972c95a282c"). InnerVolumeSpecName "kube-api-access-xdnn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:17:28.174681 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.174644    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f81322ae-0b32-434d-867a-f972c95a282c-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:17:28.174681 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.174677    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdnn4\" (UniqueName: \"kubernetes.io/projected/f81322ae-0b32-434d-867a-f972c95a282c-kube-api-access-xdnn4\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:17:28.174681 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.174687    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81322ae-0b32-434d-867a-f972c95a282c-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:17:28.691688 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.691652    2576 generic.go:356] "Generic (PLEG): container finished" podID="f81322ae-0b32-434d-867a-f972c95a282c" containerID="b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42" exitCode=0
Aug 02 11:17:28.691859 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.691707    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"
Aug 02 11:17:28.691859 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.691726    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl" event={"ID":"f81322ae-0b32-434d-867a-f972c95a282c","Type":"ContainerDied","Data":"b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42"}
Aug 02 11:17:28.691859 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.691787    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl" event={"ID":"f81322ae-0b32-434d-867a-f972c95a282c","Type":"ContainerDied","Data":"332ca37073a08feb634599abdb71a906392ecb87d3e31e88b4fd22e82a6ce60a"}
Aug 02 11:17:28.691859 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.691813    2576 scope.go:117] "RemoveContainer" containerID="b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42"
Aug 02 11:17:28.701523 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.701487    2576 scope.go:117] "RemoveContainer" containerID="b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42"
Aug 02 11:17:28.701834 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:17:28.701811    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42\": container with ID starting with b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42 not found: ID does not exist" containerID="b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42"
Aug 02 11:17:28.701915 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.701845    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42"} err="failed to get container status \"b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42\": rpc error: code = NotFound desc = could not find container \"b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42\": container with ID starting with b860573e95a81919481be4a4848aeb5a78a65b9993b0b88e1084f1d0fc72eb42 not found: ID does not exist"
Aug 02 11:17:28.716250 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.716218    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"]
Aug 02 11:17:28.719516 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.719493    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9dd0457-88fbd8d49-fx5pl"]
Aug 02 11:17:28.951125 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:17:28.951051    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f81322ae-0b32-434d-867a-f972c95a282c" path="/var/lib/kubelet/pods/f81322ae-0b32-434d-867a-f972c95a282c/volumes"
Aug 02 11:18:14.740005 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:14.739943    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"]
Aug 02 11:18:14.740649 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:14.740268    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy" containerID="cri-o://56be0e97723b403e5f2d0815a841072dfa0a8c059d3617d882fe1998304e5018" gracePeriod=30
Aug 02 11:18:17.125154 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:17.125115    2576 patch_prober.go:28] interesting pod/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 02 11:18:17.125549 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:17.125177    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 02 11:18:19.107521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:19.107491    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:18:19.136350 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:19.136324    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:18:19.976288 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:19.976256    2576 generic.go:356] "Generic (PLEG): container finished" podID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerID="56be0e97723b403e5f2d0815a841072dfa0a8c059d3617d882fe1998304e5018" exitCode=0
Aug 02 11:18:19.976414 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:19.976328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" event={"ID":"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8","Type":"ContainerDied","Data":"56be0e97723b403e5f2d0815a841072dfa0a8c059d3617d882fe1998304e5018"}
Aug 02 11:18:19.976462 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:19.976432    2576 scope.go:117] "RemoveContainer" containerID="b698903559d5810ba0705e54e9c5e10f5bab718bb41cb66b7c8e085d27a11119"
Aug 02 11:18:19.997112 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:19.997081    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:18:20.041925 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.041852    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26dq7\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-kube-api-access-26dq7\") pod \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") "
Aug 02 11:18:20.041925 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.041898    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-token\") pod \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") "
Aug 02 11:18:20.042165 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.041955    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-socket\") pod \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") "
Aug 02 11:18:20.042165 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042015    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istiod-ca-cert\") pod \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") "
Aug 02 11:18:20.042165 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042076    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-certs\") pod \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") "
Aug 02 11:18:20.042165 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042117    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-podinfo\") pod \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") "
Aug 02 11:18:20.042165 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042157    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-credential-socket\") pod \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") "
Aug 02 11:18:20.042423 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042180    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-data\") pod \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") "
Aug 02 11:18:20.042423 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042245    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-envoy\") pod \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\" (UID: \"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8\") "
Aug 02 11:18:20.042423 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042323    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" (UID: "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:18:20.042578 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042476    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" (UID: "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:18:20.042636 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042599    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:18:20.042636 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042619    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-credential-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:18:20.042839 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.042797    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" (UID: "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:18:20.044718 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.043138    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" (UID: "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:18:20.044718 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.043331    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" (UID: "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:18:20.045223 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.045195    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" (UID: "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 02 11:18:20.045223 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.045214    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" (UID: "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:18:20.045345 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.045233    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-kube-api-access-26dq7" (OuterVolumeSpecName: "kube-api-access-26dq7") pod "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" (UID: "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8"). InnerVolumeSpecName "kube-api-access-26dq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:18:20.045450 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.045436    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" (UID: "a64b08fa-fcea-4bda-99d2-d0ca5eb581b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:18:20.143307 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.143269    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-envoy\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:18:20.143307 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.143302    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26dq7\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-kube-api-access-26dq7\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:18:20.143806 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.143317    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:18:20.143806 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.143329    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istiod-ca-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:18:20.143806 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.143340    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-workload-certs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:18:20.143806 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.143351    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-podinfo\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:18:20.143806 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.143361    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8-istio-data\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:18:20.983394 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.983370    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"
Aug 02 11:18:20.983569 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.983426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2" event={"ID":"a64b08fa-fcea-4bda-99d2-d0ca5eb581b8","Type":"ContainerDied","Data":"bf972fc0b1093ff65a67ed04669130b5c5af8104f403739573621a619c6a7311"}
Aug 02 11:18:20.983569 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:20.983482    2576 scope.go:117] "RemoveContainer" containerID="56be0e97723b403e5f2d0815a841072dfa0a8c059d3617d882fe1998304e5018"
Aug 02 11:18:21.014482 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:21.014445    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"]
Aug 02 11:18:21.019577 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:21.019547    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a9dd0457-openshift-default-5bf9c56dfb-vd9j2"]
Aug 02 11:18:22.950806 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:18:22.950769    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" path="/var/lib/kubelet/pods/a64b08fa-fcea-4bda-99d2-d0ca5eb581b8/volumes"
Aug 02 11:19:21.173256 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.173221    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"]
Aug 02 11:19:21.173814 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.173793    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy"
Aug 02 11:19:21.173894 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.173818    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy"
Aug 02 11:19:21.173894 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.173852    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy"
Aug 02 11:19:21.173894 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.173861    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy"
Aug 02 11:19:21.173894 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.173877    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f81322ae-0b32-434d-867a-f972c95a282c" containerName="maas-api"
Aug 02 11:19:21.173894 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.173885    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f81322ae-0b32-434d-867a-f972c95a282c" containerName="maas-api"
Aug 02 11:19:21.174174 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.173983    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy"
Aug 02 11:19:21.174174 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.173997    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a64b08fa-fcea-4bda-99d2-d0ca5eb581b8" containerName="istio-proxy"
Aug 02 11:19:21.174174 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.174018    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f81322ae-0b32-434d-867a-f972c95a282c" containerName="maas-api"
Aug 02 11:19:21.178287 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.178268    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.181135 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.181116    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-638e7f-openshift-default-dockercfg-59hdk\""
Aug 02 11:19:21.189939 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.189915    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"]
Aug 02 11:19:21.247932 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.247898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-certs\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.248115 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.247952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istiod-ca-cert\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.248115 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.247971    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-podinfo\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.248115 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.248022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-credential-socket\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.248233 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.248106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-envoy\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.248233 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.248153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-token\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.248233 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.248219    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-socket\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.248332 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.248242    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-data\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.248332 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.248281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2dgs\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-kube-api-access-t2dgs\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.348890 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.348855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-credential-socket\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349105 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.348902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-envoy\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349105 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.348939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-token\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349105 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.348982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-socket\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349105 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-data\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349105 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2dgs\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-kube-api-access-t2dgs\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349105 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349104    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-certs\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349446 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istiod-ca-cert\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349446 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-podinfo\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349446 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-credential-socket\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349446 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349352    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-socket\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349637 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349616    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-data\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349690 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349644    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-certs\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.349896 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.349875    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istiod-ca-cert\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.351722 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.351700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-envoy\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.351926 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.351907    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-podinfo\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.356833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.356812    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-token\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.357279 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.357259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2dgs\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-kube-api-access-t2dgs\") pod \"e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") " pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.498440 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.498354    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:21.651987 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.651958    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"]
Aug 02 11:19:21.653672 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:19:21.653644    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38fa15d_6f6f_4a3e_bf85_ed2deeb79c50.slice/crio-a1c38e2e3d7ada4679649feda4bc81545909078850adbd34c8fedb19b1d4e821 WatchSource:0}: Error finding container a1c38e2e3d7ada4679649feda4bc81545909078850adbd34c8fedb19b1d4e821: Status 404 returned error can't find the container with id a1c38e2e3d7ada4679649feda4bc81545909078850adbd34c8fedb19b1d4e821
Aug 02 11:19:21.655878 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.655843    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:19:21.655963 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.655921    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:19:21.656002 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:21.655959    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:19:22.337438 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:22.337399    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" event={"ID":"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50","Type":"ContainerStarted","Data":"46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225"}
Aug 02 11:19:22.337947 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:22.337464    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" event={"ID":"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50","Type":"ContainerStarted","Data":"a1c38e2e3d7ada4679649feda4bc81545909078850adbd34c8fedb19b1d4e821"}
Aug 02 11:19:22.356592 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:22.356546    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" podStartSLOduration=1.356532029 podStartE2EDuration="1.356532029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:19:22.355656113 +0000 UTC m=+2763.886238761" watchObservedRunningTime="2026-08-02 11:19:22.356532029 +0000 UTC m=+2763.887114657"
Aug 02 11:19:22.499286 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:22.499245    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:22.505530 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:22.505500    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:23.343273 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:23.343242    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:23.344264 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:23.344247    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:25.316956 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.316917    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"]
Aug 02 11:19:25.321512 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.321488    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.324119 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.324099    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-638e7f-serving-cert\""
Aug 02 11:19:25.328442 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.328418    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"]
Aug 02 11:19:25.386620 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.386589    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7kd8\" (UniqueName: \"kubernetes.io/projected/c10b7f74-8687-4785-bb65-9c8cb817479a-kube-api-access-q7kd8\") pod \"maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.386878 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.386852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10b7f74-8687-4785-bb65-9c8cb817479a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.386945 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.386930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10b7f74-8687-4785-bb65-9c8cb817479a-maas-api-tls\") pod \"maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.488385 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.488351    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7kd8\" (UniqueName: \"kubernetes.io/projected/c10b7f74-8687-4785-bb65-9c8cb817479a-kube-api-access-q7kd8\") pod \"maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.488592 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.488400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10b7f74-8687-4785-bb65-9c8cb817479a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.488592 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.488443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10b7f74-8687-4785-bb65-9c8cb817479a-maas-api-tls\") pod \"maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.489133 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.489103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10b7f74-8687-4785-bb65-9c8cb817479a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.491151 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.491131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10b7f74-8687-4785-bb65-9c8cb817479a-maas-api-tls\") pod \"maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.496308 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.496289    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7kd8\" (UniqueName: \"kubernetes.io/projected/c10b7f74-8687-4785-bb65-9c8cb817479a-kube-api-access-q7kd8\") pod \"maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:25.636684 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:25.636595    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:26.024309 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:26.024274    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"]
Aug 02 11:19:26.026289 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:19:26.026257    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10b7f74_8687_4785_bb65_9c8cb817479a.slice/crio-79e66db55a13abe703f06f22324414be9755a760b9b03293125d10b48991e1ce WatchSource:0}: Error finding container 79e66db55a13abe703f06f22324414be9755a760b9b03293125d10b48991e1ce: Status 404 returned error can't find the container with id 79e66db55a13abe703f06f22324414be9755a760b9b03293125d10b48991e1ce
Aug 02 11:19:26.027671 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:26.027648    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 02 11:19:26.361398 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:26.361312    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml" event={"ID":"c10b7f74-8687-4785-bb65-9c8cb817479a","Type":"ContainerStarted","Data":"79e66db55a13abe703f06f22324414be9755a760b9b03293125d10b48991e1ce"}
Aug 02 11:19:27.367991 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:27.367950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml" event={"ID":"c10b7f74-8687-4785-bb65-9c8cb817479a","Type":"ContainerStarted","Data":"31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992"}
Aug 02 11:19:27.368441 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:27.368082    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:27.383698 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:27.383652    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml" podStartSLOduration=1.924013783 podStartE2EDuration="2.383636191s" totalImagesPullingTime="459.622408ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:19:27.382973315 +0000 UTC m=+2768.913555965" watchObservedRunningTime="2026-08-02 11:19:27.383636191 +0000 UTC m=+2768.914218820"
Aug 02 11:19:33.379875 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:33.379844    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:19:37.441217 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:37.441180    2576 generic.go:356] "Generic (PLEG): container finished" podID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerID="46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225" exitCode=0
Aug 02 11:19:37.441629 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:37.441251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" event={"ID":"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50","Type":"ContainerDied","Data":"46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225"}
Aug 02 11:19:37.441674 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:37.441634    2576 scope.go:117] "RemoveContainer" containerID="46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225"
Aug 02 11:19:37.442603 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:37.442579    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:19:37.442654 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:37.442644    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:19:37.442693 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:37.442673    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:19:38.343952 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:38.343913    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:38.449063 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:38.449013    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" event={"ID":"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50","Type":"ContainerStarted","Data":"589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321"}
Aug 02 11:19:38.499004 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:38.498965    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:38.500878 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:38.500834    2576 patch_prober.go:28] interesting pod/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Aug 02 11:19:38.501075 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:38.500910    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Aug 02 11:19:39.499349 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:39.499261    2576 patch_prober.go:28] interesting pod/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Aug 02 11:19:39.499910 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:39.499878    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Aug 02 11:19:40.498781 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:40.498741    2576 patch_prober.go:28] interesting pod/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Aug 02 11:19:40.499110 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:40.499080    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Aug 02 11:19:41.499293 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:41.499248    2576 patch_prober.go:28] interesting pod/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Aug 02 11:19:41.499807 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:41.499330    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Aug 02 11:19:42.499590 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:42.499544    2576 patch_prober.go:28] interesting pod/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Aug 02 11:19:42.500110 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:42.499618    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Aug 02 11:19:43.499294 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:43.499260    2576 patch_prober.go:28] interesting pod/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Aug 02 11:19:43.499486 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:43.499315    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Aug 02 11:19:44.503616 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:44.503586    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:44.504023 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:44.503929    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:44.504675 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:44.504658    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:19:53.441846 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.441810    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"]
Aug 02 11:19:53.446311 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.446288    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.449016 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.448936    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-66e93e-serving-cert\""
Aug 02 11:19:53.457135 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.457092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"]
Aug 02 11:19:53.546150 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.546118    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-maas-api-tls\") pod \"maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.546322 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.546160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.546322 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.546259    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trcqn\" (UniqueName: \"kubernetes.io/projected/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-kube-api-access-trcqn\") pod \"maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.646944 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.646907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-maas-api-tls\") pod \"maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.646944 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.646949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.647204 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.646995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trcqn\" (UniqueName: \"kubernetes.io/projected/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-kube-api-access-trcqn\") pod \"maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.648142 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.648111    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.649806 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.649779    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-maas-api-tls\") pod \"maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.655421 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.655398    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trcqn\" (UniqueName: \"kubernetes.io/projected/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-kube-api-access-trcqn\") pod \"maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.763145 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.763062    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:53.909627 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:53.909541    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"]
Aug 02 11:19:53.910213 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:19:53.910174    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6cc356_8f49_4ab3_9274_d3bac8b84fd5.slice/crio-3f67528944d3c7f1d363797726377dafeda4a193fb04e3d7812691eec682a6ba WatchSource:0}: Error finding container 3f67528944d3c7f1d363797726377dafeda4a193fb04e3d7812691eec682a6ba: Status 404 returned error can't find the container with id 3f67528944d3c7f1d363797726377dafeda4a193fb04e3d7812691eec682a6ba
Aug 02 11:19:54.540244 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:54.540152    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm" event={"ID":"af6cc356-8f49-4ab3-9274-d3bac8b84fd5","Type":"ContainerStarted","Data":"1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33"}
Aug 02 11:19:54.540244 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:54.540214    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm" event={"ID":"af6cc356-8f49-4ab3-9274-d3bac8b84fd5","Type":"ContainerStarted","Data":"3f67528944d3c7f1d363797726377dafeda4a193fb04e3d7812691eec682a6ba"}
Aug 02 11:19:54.540664 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:54.540352    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:19:54.555848 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:19:54.555793    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm" podStartSLOduration=1.241258828 podStartE2EDuration="1.555773897s" totalImagesPullingTime="314.515069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:19:54.555641146 +0000 UTC m=+2796.086223804" watchObservedRunningTime="2026-08-02 11:19:54.555773897 +0000 UTC m=+2796.086356529"
Aug 02 11:20:00.552536 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:20:00.552503    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:21:58.786630 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:58.786593    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"]
Aug 02 11:21:58.787189 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:58.786824    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml" podUID="c10b7f74-8687-4785-bb65-9c8cb817479a" containerName="maas-api" containerID="cri-o://31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992" gracePeriod=30
Aug 02 11:21:59.065401 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.065378    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:21:59.169458 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.169432    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7kd8\" (UniqueName: \"kubernetes.io/projected/c10b7f74-8687-4785-bb65-9c8cb817479a-kube-api-access-q7kd8\") pod \"c10b7f74-8687-4785-bb65-9c8cb817479a\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") "
Aug 02 11:21:59.169603 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.169555    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10b7f74-8687-4785-bb65-9c8cb817479a-openshift-service-ca\") pod \"c10b7f74-8687-4785-bb65-9c8cb817479a\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") "
Aug 02 11:21:59.169603 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.169596    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10b7f74-8687-4785-bb65-9c8cb817479a-maas-api-tls\") pod \"c10b7f74-8687-4785-bb65-9c8cb817479a\" (UID: \"c10b7f74-8687-4785-bb65-9c8cb817479a\") "
Aug 02 11:21:59.169964 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.169933    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c10b7f74-8687-4785-bb65-9c8cb817479a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c10b7f74-8687-4785-bb65-9c8cb817479a" (UID: "c10b7f74-8687-4785-bb65-9c8cb817479a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:21:59.171718 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.171684    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c10b7f74-8687-4785-bb65-9c8cb817479a-kube-api-access-q7kd8" (OuterVolumeSpecName: "kube-api-access-q7kd8") pod "c10b7f74-8687-4785-bb65-9c8cb817479a" (UID: "c10b7f74-8687-4785-bb65-9c8cb817479a"). InnerVolumeSpecName "kube-api-access-q7kd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:21:59.171808 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.171693    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c10b7f74-8687-4785-bb65-9c8cb817479a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c10b7f74-8687-4785-bb65-9c8cb817479a" (UID: "c10b7f74-8687-4785-bb65-9c8cb817479a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:21:59.264575 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.264547    2576 generic.go:356] "Generic (PLEG): container finished" podID="c10b7f74-8687-4785-bb65-9c8cb817479a" containerID="31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992" exitCode=0
Aug 02 11:21:59.264719 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.264592    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"
Aug 02 11:21:59.264719 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.264639    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml" event={"ID":"c10b7f74-8687-4785-bb65-9c8cb817479a","Type":"ContainerDied","Data":"31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992"}
Aug 02 11:21:59.264719 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.264691    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml" event={"ID":"c10b7f74-8687-4785-bb65-9c8cb817479a","Type":"ContainerDied","Data":"79e66db55a13abe703f06f22324414be9755a760b9b03293125d10b48991e1ce"}
Aug 02 11:21:59.264719 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.264715    2576 scope.go:117] "RemoveContainer" containerID="31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992"
Aug 02 11:21:59.270411 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.270384    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7kd8\" (UniqueName: \"kubernetes.io/projected/c10b7f74-8687-4785-bb65-9c8cb817479a-kube-api-access-q7kd8\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:21:59.270547 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.270416    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10b7f74-8687-4785-bb65-9c8cb817479a-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:21:59.270547 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.270431    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10b7f74-8687-4785-bb65-9c8cb817479a-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:21:59.274572 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.274555    2576 scope.go:117] "RemoveContainer" containerID="31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992"
Aug 02 11:21:59.274837 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:21:59.274818    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992\": container with ID starting with 31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992 not found: ID does not exist" containerID="31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992"
Aug 02 11:21:59.274931 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.274847    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992"} err="failed to get container status \"31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992\": rpc error: code = NotFound desc = could not find container \"31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992\": container with ID starting with 31205cb827edd87b607a5deef78605304a1c14062464d1de86f2ec8a63114992 not found: ID does not exist"
Aug 02 11:21:59.287162 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.287141    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"]
Aug 02 11:21:59.292920 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:21:59.292897    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-638e7f-7b9774885b-bfzml"]
Aug 02 11:22:00.951119 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:00.951081    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c10b7f74-8687-4785-bb65-9c8cb817479a" path="/var/lib/kubelet/pods/c10b7f74-8687-4785-bb65-9c8cb817479a/volumes"
Aug 02 11:22:41.564214 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:41.564137    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"]
Aug 02 11:22:41.564687 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:41.564422    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy" containerID="cri-o://589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321" gracePeriod=30
Aug 02 11:22:42.638898 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:42.638865    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"]
Aug 02 11:22:42.639369 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:42.639090    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm" podUID="af6cc356-8f49-4ab3-9274-d3bac8b84fd5" containerName="maas-api" containerID="cri-o://1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33" gracePeriod=30
Aug 02 11:22:42.902443 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:42.902422    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:22:42.955710 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:42.955678    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trcqn\" (UniqueName: \"kubernetes.io/projected/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-kube-api-access-trcqn\") pod \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") "
Aug 02 11:22:42.955852 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:42.955731    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-openshift-service-ca\") pod \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") "
Aug 02 11:22:42.955904 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:42.955847    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-maas-api-tls\") pod \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\" (UID: \"af6cc356-8f49-4ab3-9274-d3bac8b84fd5\") "
Aug 02 11:22:42.956130 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:42.956103    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af6cc356-8f49-4ab3-9274-d3bac8b84fd5" (UID: "af6cc356-8f49-4ab3-9274-d3bac8b84fd5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:22:42.957886 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:42.957861    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-kube-api-access-trcqn" (OuterVolumeSpecName: "kube-api-access-trcqn") pod "af6cc356-8f49-4ab3-9274-d3bac8b84fd5" (UID: "af6cc356-8f49-4ab3-9274-d3bac8b84fd5"). InnerVolumeSpecName "kube-api-access-trcqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:22:42.957980 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:42.957955    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af6cc356-8f49-4ab3-9274-d3bac8b84fd5" (UID: "af6cc356-8f49-4ab3-9274-d3bac8b84fd5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:22:43.057467 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.057434    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:43.057467 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.057463    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trcqn\" (UniqueName: \"kubernetes.io/projected/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-kube-api-access-trcqn\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:43.057467 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.057474    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af6cc356-8f49-4ab3-9274-d3bac8b84fd5-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:43.515325 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.515290    2576 generic.go:356] "Generic (PLEG): container finished" podID="af6cc356-8f49-4ab3-9274-d3bac8b84fd5" containerID="1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33" exitCode=0
Aug 02 11:22:43.515516 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.515327    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm" event={"ID":"af6cc356-8f49-4ab3-9274-d3bac8b84fd5","Type":"ContainerDied","Data":"1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33"}
Aug 02 11:22:43.515516 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.515340    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"
Aug 02 11:22:43.515516 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.515394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm" event={"ID":"af6cc356-8f49-4ab3-9274-d3bac8b84fd5","Type":"ContainerDied","Data":"3f67528944d3c7f1d363797726377dafeda4a193fb04e3d7812691eec682a6ba"}
Aug 02 11:22:43.515516 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.515424    2576 scope.go:117] "RemoveContainer" containerID="1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33"
Aug 02 11:22:43.525586 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.525570    2576 scope.go:117] "RemoveContainer" containerID="1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33"
Aug 02 11:22:43.525854 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:22:43.525835    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33\": container with ID starting with 1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33 not found: ID does not exist" containerID="1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33"
Aug 02 11:22:43.525926 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.525860    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33"} err="failed to get container status \"1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33\": rpc error: code = NotFound desc = could not find container \"1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33\": container with ID starting with 1f3934668ed155adf7d590f53b87b06faa02218225422874c8a555c170768d33 not found: ID does not exist"
Aug 02 11:22:43.537848 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.537825    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"]
Aug 02 11:22:43.540149 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:43.540124    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-66e93e-66dff96b6d-jrtrm"]
Aug 02 11:22:44.505889 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:44.505847    2576 patch_prober.go:28] interesting pod/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 02 11:22:44.506281 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:44.505916    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 02 11:22:44.950424 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:44.950396    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af6cc356-8f49-4ab3-9274-d3bac8b84fd5" path="/var/lib/kubelet/pods/af6cc356-8f49-4ab3-9274-d3bac8b84fd5/volumes"
Aug 02 11:22:46.818432 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.818408    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:22:46.887904 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.887815    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istiod-ca-cert\") pod \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") "
Aug 02 11:22:46.887904 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.887853    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-podinfo\") pod \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") "
Aug 02 11:22:46.887904 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.887881    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-certs\") pod \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") "
Aug 02 11:22:46.887904 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.887905    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-credential-socket\") pod \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") "
Aug 02 11:22:46.888272 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.887941    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-token\") pod \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") "
Aug 02 11:22:46.888272 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.887974    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-data\") pod \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") "
Aug 02 11:22:46.888272 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.888017    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-socket\") pod \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") "
Aug 02 11:22:46.888272 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.888053    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2dgs\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-kube-api-access-t2dgs\") pod \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") "
Aug 02 11:22:46.888272 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.888097    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-envoy\") pod \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\" (UID: \"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50\") "
Aug 02 11:22:46.888539 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.888292    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-data" (OuterVolumeSpecName: "istio-data") pod "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" (UID: "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:22:46.888539 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.888448    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" (UID: "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:22:46.888539 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.888488    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" (UID: "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:22:46.888539 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.888523    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" (UID: "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:22:46.888539 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.888550    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-data\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:46.888846 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.888571    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" (UID: "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:22:46.890598 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.890575    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" (UID: "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:22:46.890741 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.890609    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" (UID: "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 02 11:22:46.891019 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.891004    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-kube-api-access-t2dgs" (OuterVolumeSpecName: "kube-api-access-t2dgs") pod "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" (UID: "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50"). InnerVolumeSpecName "kube-api-access-t2dgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:22:46.891116 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.891012    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-token" (OuterVolumeSpecName: "istio-token") pod "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" (UID: "d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:22:46.989561 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.989535    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2dgs\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-kube-api-access-t2dgs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:46.989561 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.989559    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-envoy\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:46.989709 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.989568    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istiod-ca-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:46.989709 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.989576    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-podinfo\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:46.989709 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.989584    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-certs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:46.989709 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.989592    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-credential-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:46.989709 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.989600    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-istio-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:46.989709 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:46.989610    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50-workload-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:22:47.539483 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.539449    2576 generic.go:356] "Generic (PLEG): container finished" podID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerID="589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321" exitCode=0
Aug 02 11:22:47.539681 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.539491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" event={"ID":"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50","Type":"ContainerDied","Data":"589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321"}
Aug 02 11:22:47.539681 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.539526    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"
Aug 02 11:22:47.539681 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.539593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5" event={"ID":"d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50","Type":"ContainerDied","Data":"a1c38e2e3d7ada4679649feda4bc81545909078850adbd34c8fedb19b1d4e821"}
Aug 02 11:22:47.539681 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.539625    2576 scope.go:117] "RemoveContainer" containerID="589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321"
Aug 02 11:22:47.548789 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.548775    2576 scope.go:117] "RemoveContainer" containerID="46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225"
Aug 02 11:22:47.558264 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.558248    2576 scope.go:117] "RemoveContainer" containerID="589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321"
Aug 02 11:22:47.558488 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:22:47.558467    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321\": container with ID starting with 589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321 not found: ID does not exist" containerID="589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321"
Aug 02 11:22:47.558550 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.558504    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"]
Aug 02 11:22:47.558550 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.558501    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321"} err="failed to get container status \"589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321\": rpc error: code = NotFound desc = could not find container \"589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321\": container with ID starting with 589ae89b722826928b74b8b29266ac5f4c2e32393eb0bfd0be3de9f4225e7321 not found: ID does not exist"
Aug 02 11:22:47.558639 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.558551    2576 scope.go:117] "RemoveContainer" containerID="46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225"
Aug 02 11:22:47.558785 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:22:47.558768    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225\": container with ID starting with 46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225 not found: ID does not exist" containerID="46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225"
Aug 02 11:22:47.558830 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.558792    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225"} err="failed to get container status \"46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225\": rpc error: code = NotFound desc = could not find container \"46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225\": container with ID starting with 46c0c7750e1bab66653345a22633f8414c55af2eef3a8f4675213825a7b13225 not found: ID does not exist"
Aug 02 11:22:47.562820 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:47.562799    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-638e7f-openshift-default-654bc46fb4-l5rh5"]
Aug 02 11:22:48.952159 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:48.952126    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" path="/var/lib/kubelet/pods/d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50/volumes"
Aug 02 11:22:50.307065 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307012    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"]
Aug 02 11:22:50.307642 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307626    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c10b7f74-8687-4785-bb65-9c8cb817479a" containerName="maas-api"
Aug 02 11:22:50.307728 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307645    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c10b7f74-8687-4785-bb65-9c8cb817479a" containerName="maas-api"
Aug 02 11:22:50.307728 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307661    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy"
Aug 02 11:22:50.307728 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307670    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy"
Aug 02 11:22:50.307728 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307682    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy"
Aug 02 11:22:50.307728 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307691    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy"
Aug 02 11:22:50.307728 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307699    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af6cc356-8f49-4ab3-9274-d3bac8b84fd5" containerName="maas-api"
Aug 02 11:22:50.307728 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307707    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="af6cc356-8f49-4ab3-9274-d3bac8b84fd5" containerName="maas-api"
Aug 02 11:22:50.308017 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307791    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c10b7f74-8687-4785-bb65-9c8cb817479a" containerName="maas-api"
Aug 02 11:22:50.308017 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307802    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy"
Aug 02 11:22:50.308017 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307813    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38fa15d-6f6f-4a3e-bf85-ed2deeb79c50" containerName="istio-proxy"
Aug 02 11:22:50.308017 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.307825    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="af6cc356-8f49-4ab3-9274-d3bac8b84fd5" containerName="maas-api"
Aug 02 11:22:50.312751 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.312730    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.315377 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.315358    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8cfbc4-openshift-default-dockercfg-nc6nl\""
Aug 02 11:22:50.320571 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.320546    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"]
Aug 02 11:22:50.429611 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.429578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-socket\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.429772 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.429621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-certs\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.429772 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.429644    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af13a3b-a68f-4156-9892-cb16249da39a-istio-podinfo\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.429772 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.429707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-credential-socket\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.429772 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.429758    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-istio-token\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.429909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.429789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2bh4\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-kube-api-access-t2bh4\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.429909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.429815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af13a3b-a68f-4156-9892-cb16249da39a-istiod-ca-cert\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.429909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.429835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-envoy\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.429909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.429864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-data\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.530993 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.530955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-envoy\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.530993 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.530999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-data\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531289 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531076    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-socket\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531289 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-certs\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531289 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531162    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af13a3b-a68f-4156-9892-cb16249da39a-istio-podinfo\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531289 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-credential-socket\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531289 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-istio-token\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531561 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2bh4\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-kube-api-access-t2bh4\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531561 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af13a3b-a68f-4156-9892-cb16249da39a-istiod-ca-cert\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531561 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-data\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531561 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531501    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-socket\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531836 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531592    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-certs\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.531836 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.531632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-credential-socket\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.532059 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.532015    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af13a3b-a68f-4156-9892-cb16249da39a-istiod-ca-cert\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.533676 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.533653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-envoy\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.533805 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.533781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af13a3b-a68f-4156-9892-cb16249da39a-istio-podinfo\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.539121 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.539100    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2bh4\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-kube-api-access-t2bh4\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.539611 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.539587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-istio-token\") pod \"e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") " pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.625150 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.625074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:50.775975 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.775948    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"]
Aug 02 11:22:50.777813 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:22:50.777792    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af13a3b_a68f_4156_9892_cb16249da39a.slice/crio-29b775b87bb2891965ddaf230fe9e16df76d51f876eb6f6fe8f5e3dd3a06d234 WatchSource:0}: Error finding container 29b775b87bb2891965ddaf230fe9e16df76d51f876eb6f6fe8f5e3dd3a06d234: Status 404 returned error can't find the container with id 29b775b87bb2891965ddaf230fe9e16df76d51f876eb6f6fe8f5e3dd3a06d234
Aug 02 11:22:50.779859 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.779827    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:22:50.779947 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.779887    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:22:50.779947 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:50.779914    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:22:51.566594 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:51.566554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" event={"ID":"2af13a3b-a68f-4156-9892-cb16249da39a","Type":"ContainerStarted","Data":"e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba"}
Aug 02 11:22:51.567014 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:51.566615    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" event={"ID":"2af13a3b-a68f-4156-9892-cb16249da39a","Type":"ContainerStarted","Data":"29b775b87bb2891965ddaf230fe9e16df76d51f876eb6f6fe8f5e3dd3a06d234"}
Aug 02 11:22:51.589541 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:51.589499    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" podStartSLOduration=1.5894851669999999 podStartE2EDuration="1.589485167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:22:51.585933767 +0000 UTC m=+2973.116516394" watchObservedRunningTime="2026-08-02 11:22:51.589485167 +0000 UTC m=+2973.120067797"
Aug 02 11:22:51.626021 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:51.625995    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:51.631054 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:51.631007    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:52.572356 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:52.572325    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:52.573324 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:52.573303    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:22:54.540634 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.540597    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"]
Aug 02 11:22:54.544624 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.544602    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.547083 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.547020    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-8cfbc4-serving-cert\""
Aug 02 11:22:54.552627 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.552489    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"]
Aug 02 11:22:54.668273 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.668230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e6bbe6-5e69-496d-abfa-36930e82ca98-maas-api-tls\") pod \"maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.668488 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.668322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e6bbe6-5e69-496d-abfa-36930e82ca98-openshift-service-ca\") pod \"maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.668488 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.668400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgkmq\" (UniqueName: \"kubernetes.io/projected/c4e6bbe6-5e69-496d-abfa-36930e82ca98-kube-api-access-fgkmq\") pod \"maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.769726 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.769684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e6bbe6-5e69-496d-abfa-36930e82ca98-maas-api-tls\") pod \"maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.769902 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.769754    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e6bbe6-5e69-496d-abfa-36930e82ca98-openshift-service-ca\") pod \"maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.769902 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.769814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgkmq\" (UniqueName: \"kubernetes.io/projected/c4e6bbe6-5e69-496d-abfa-36930e82ca98-kube-api-access-fgkmq\") pod \"maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.770418 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.770399    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e6bbe6-5e69-496d-abfa-36930e82ca98-openshift-service-ca\") pod \"maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.772345 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.772313    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e6bbe6-5e69-496d-abfa-36930e82ca98-maas-api-tls\") pod \"maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.778415 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.778393    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgkmq\" (UniqueName: \"kubernetes.io/projected/c4e6bbe6-5e69-496d-abfa-36930e82ca98-kube-api-access-fgkmq\") pod \"maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:54.860073 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:54.859955    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:55.014673 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:55.014639    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"]
Aug 02 11:22:55.016801 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:22:55.016776    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e6bbe6_5e69_496d_abfa_36930e82ca98.slice/crio-3c2921e2ba36f3baddcc799ce1978f10da0e97871820c5e40aed2b2a586c0543 WatchSource:0}: Error finding container 3c2921e2ba36f3baddcc799ce1978f10da0e97871820c5e40aed2b2a586c0543: Status 404 returned error can't find the container with id 3c2921e2ba36f3baddcc799ce1978f10da0e97871820c5e40aed2b2a586c0543
Aug 02 11:22:55.593010 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:55.592963    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s" event={"ID":"c4e6bbe6-5e69-496d-abfa-36930e82ca98","Type":"ContainerStarted","Data":"985ed7a1438f3417930adddd3800dc761ecf44676dc6b1df5328cc91b1e7b5c7"}
Aug 02 11:22:55.593437 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:55.593041    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s" event={"ID":"c4e6bbe6-5e69-496d-abfa-36930e82ca98","Type":"ContainerStarted","Data":"3c2921e2ba36f3baddcc799ce1978f10da0e97871820c5e40aed2b2a586c0543"}
Aug 02 11:22:55.593437 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:55.593095    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:22:55.612492 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:22:55.612438    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s" podStartSLOduration=1.200256502 podStartE2EDuration="1.612422511s" totalImagesPullingTime="412.166009ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:22:55.609969006 +0000 UTC m=+2977.140551636" watchObservedRunningTime="2026-08-02 11:22:55.612422511 +0000 UTC m=+2977.143005185"
Aug 02 11:23:01.611054 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:01.610991    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:23:06.669650 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:06.669615    2576 generic.go:356] "Generic (PLEG): container finished" podID="2af13a3b-a68f-4156-9892-cb16249da39a" containerID="e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba" exitCode=0
Aug 02 11:23:06.670088 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:06.669684    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" event={"ID":"2af13a3b-a68f-4156-9892-cb16249da39a","Type":"ContainerDied","Data":"e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba"}
Aug 02 11:23:06.670139 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:06.670081    2576 scope.go:117] "RemoveContainer" containerID="e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba"
Aug 02 11:23:06.671139 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:06.671112    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:23:06.671213 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:06.671182    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:23:06.671250 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:06.671221    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:23:07.572801 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:07.572758    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:23:07.677304 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:07.677266    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" event={"ID":"2af13a3b-a68f-4156-9892-cb16249da39a","Type":"ContainerStarted","Data":"8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7"}
Aug 02 11:23:08.626014 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:08.625984    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:23:08.627221 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:08.627198    2576 patch_prober.go:28] interesting pod/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Aug 02 11:23:08.627283 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:08.627250    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Aug 02 11:23:09.625507 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:09.625463    2576 patch_prober.go:28] interesting pod/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Aug 02 11:23:09.625954 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:09.625518    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Aug 02 11:23:10.625414 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:10.625377    2576 patch_prober.go:28] interesting pod/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Aug 02 11:23:10.625583 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:10.625438    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Aug 02 11:23:11.625625 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:11.625584    2576 patch_prober.go:28] interesting pod/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Aug 02 11:23:11.626012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:11.625657    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Aug 02 11:23:12.626418 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:12.626376    2576 patch_prober.go:28] interesting pod/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Aug 02 11:23:12.626796 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:12.626463    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Aug 02 11:23:13.629258 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:13.629228    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:23:13.629637 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:13.629503    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:23:13.630168 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:13.630154    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:23:19.162748 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:19.162717    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:23:19.185722 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:19.185691    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:23:23.346216 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.346170    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"]
Aug 02 11:23:23.351039 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.351004    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.353500 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.353480    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-5afe3e-serving-cert\""
Aug 02 11:23:23.363443 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.363411    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"]
Aug 02 11:23:23.464382 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.464348    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c899af30-26f0-4b2a-8453-0ba9df9a0991-maas-api-tls\") pod \"maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.464548 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.464405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c899af30-26f0-4b2a-8453-0ba9df9a0991-openshift-service-ca\") pod \"maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.464548 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.464482    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6rkr\" (UniqueName: \"kubernetes.io/projected/c899af30-26f0-4b2a-8453-0ba9df9a0991-kube-api-access-v6rkr\") pod \"maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.565484 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.565447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c899af30-26f0-4b2a-8453-0ba9df9a0991-openshift-service-ca\") pod \"maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.565658 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.565536    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6rkr\" (UniqueName: \"kubernetes.io/projected/c899af30-26f0-4b2a-8453-0ba9df9a0991-kube-api-access-v6rkr\") pod \"maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.565658 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.565601    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c899af30-26f0-4b2a-8453-0ba9df9a0991-maas-api-tls\") pod \"maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.566264 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.566244    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c899af30-26f0-4b2a-8453-0ba9df9a0991-openshift-service-ca\") pod \"maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.568183 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.568162    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c899af30-26f0-4b2a-8453-0ba9df9a0991-maas-api-tls\") pod \"maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.576004 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.575982    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6rkr\" (UniqueName: \"kubernetes.io/projected/c899af30-26f0-4b2a-8453-0ba9df9a0991-kube-api-access-v6rkr\") pod \"maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.667159 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.667118    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:23.818890 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:23.818860    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"]
Aug 02 11:23:24.784387 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:24.784354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq" event={"ID":"c899af30-26f0-4b2a-8453-0ba9df9a0991","Type":"ContainerStarted","Data":"01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d"}
Aug 02 11:23:24.784875 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:24.784407    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq" event={"ID":"c899af30-26f0-4b2a-8453-0ba9df9a0991","Type":"ContainerStarted","Data":"97f84aafc61c768f4dceb3baa90d502277caf709db851c697b8243fd848d000b"}
Aug 02 11:23:24.784875 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:24.784480    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:24.802391 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:24.802341    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq" podStartSLOduration=1.463088415 podStartE2EDuration="1.80232112s" totalImagesPullingTime="339.232705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:23:24.799789766 +0000 UTC m=+3006.330372434" watchObservedRunningTime="2026-08-02 11:23:24.80232112 +0000 UTC m=+3006.332903749"
Aug 02 11:23:30.795842 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:30.795813    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:23:55.687407 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:55.687365    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"]
Aug 02 11:23:55.688063 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:55.687644    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s" podUID="c4e6bbe6-5e69-496d-abfa-36930e82ca98" containerName="maas-api" containerID="cri-o://985ed7a1438f3417930adddd3800dc761ecf44676dc6b1df5328cc91b1e7b5c7" gracePeriod=30
Aug 02 11:23:55.969362 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:55.969335    2576 generic.go:356] "Generic (PLEG): container finished" podID="c4e6bbe6-5e69-496d-abfa-36930e82ca98" containerID="985ed7a1438f3417930adddd3800dc761ecf44676dc6b1df5328cc91b1e7b5c7" exitCode=0
Aug 02 11:23:55.969493 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:55.969398    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s" event={"ID":"c4e6bbe6-5e69-496d-abfa-36930e82ca98","Type":"ContainerDied","Data":"985ed7a1438f3417930adddd3800dc761ecf44676dc6b1df5328cc91b1e7b5c7"}
Aug 02 11:23:55.969493 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:55.969453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s" event={"ID":"c4e6bbe6-5e69-496d-abfa-36930e82ca98","Type":"ContainerDied","Data":"3c2921e2ba36f3baddcc799ce1978f10da0e97871820c5e40aed2b2a586c0543"}
Aug 02 11:23:55.969493 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:55.969471    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c2921e2ba36f3baddcc799ce1978f10da0e97871820c5e40aed2b2a586c0543"
Aug 02 11:23:55.970715 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:55.970691    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:23:56.162335 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.162302    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgkmq\" (UniqueName: \"kubernetes.io/projected/c4e6bbe6-5e69-496d-abfa-36930e82ca98-kube-api-access-fgkmq\") pod \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") "
Aug 02 11:23:56.162500 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.162345    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e6bbe6-5e69-496d-abfa-36930e82ca98-openshift-service-ca\") pod \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") "
Aug 02 11:23:56.162500 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.162432    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e6bbe6-5e69-496d-abfa-36930e82ca98-maas-api-tls\") pod \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\" (UID: \"c4e6bbe6-5e69-496d-abfa-36930e82ca98\") "
Aug 02 11:23:56.162798 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.162767    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4e6bbe6-5e69-496d-abfa-36930e82ca98-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4e6bbe6-5e69-496d-abfa-36930e82ca98" (UID: "c4e6bbe6-5e69-496d-abfa-36930e82ca98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:23:56.164521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.164496    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4e6bbe6-5e69-496d-abfa-36930e82ca98-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4e6bbe6-5e69-496d-abfa-36930e82ca98" (UID: "c4e6bbe6-5e69-496d-abfa-36930e82ca98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:23:56.164706 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.164682    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e6bbe6-5e69-496d-abfa-36930e82ca98-kube-api-access-fgkmq" (OuterVolumeSpecName: "kube-api-access-fgkmq") pod "c4e6bbe6-5e69-496d-abfa-36930e82ca98" (UID: "c4e6bbe6-5e69-496d-abfa-36930e82ca98"). InnerVolumeSpecName "kube-api-access-fgkmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:23:56.264173 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.264097    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4e6bbe6-5e69-496d-abfa-36930e82ca98-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:23:56.264173 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.264121    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgkmq\" (UniqueName: \"kubernetes.io/projected/c4e6bbe6-5e69-496d-abfa-36930e82ca98-kube-api-access-fgkmq\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:23:56.264173 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.264130    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4e6bbe6-5e69-496d-abfa-36930e82ca98-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:23:56.974125 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.974097    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"
Aug 02 11:23:56.990452 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.990422    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"]
Aug 02 11:23:56.995837 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:56.995812    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8cfbc4-5df9b668c4-8s95s"]
Aug 02 11:23:58.958329 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:23:58.958295    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4e6bbe6-5e69-496d-abfa-36930e82ca98" path="/var/lib/kubelet/pods/c4e6bbe6-5e69-496d-abfa-36930e82ca98/volumes"
Aug 02 11:24:02.264099 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:02.264063    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"]
Aug 02 11:24:02.264569 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:02.264303    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy" containerID="cri-o://8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7" gracePeriod=30
Aug 02 11:24:07.521540 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.521517    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:24:07.552610 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552543    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af13a3b-a68f-4156-9892-cb16249da39a-istio-podinfo\") pod \"2af13a3b-a68f-4156-9892-cb16249da39a\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") "
Aug 02 11:24:07.552610 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552589    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-data\") pod \"2af13a3b-a68f-4156-9892-cb16249da39a\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") "
Aug 02 11:24:07.552794 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552632    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-socket\") pod \"2af13a3b-a68f-4156-9892-cb16249da39a\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") "
Aug 02 11:24:07.552794 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552673    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2bh4\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-kube-api-access-t2bh4\") pod \"2af13a3b-a68f-4156-9892-cb16249da39a\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") "
Aug 02 11:24:07.552794 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552704    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af13a3b-a68f-4156-9892-cb16249da39a-istiod-ca-cert\") pod \"2af13a3b-a68f-4156-9892-cb16249da39a\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") "
Aug 02 11:24:07.552794 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552759    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-istio-token\") pod \"2af13a3b-a68f-4156-9892-cb16249da39a\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") "
Aug 02 11:24:07.552794 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552785    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-envoy\") pod \"2af13a3b-a68f-4156-9892-cb16249da39a\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") "
Aug 02 11:24:07.553077 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552830    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-credential-socket\") pod \"2af13a3b-a68f-4156-9892-cb16249da39a\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") "
Aug 02 11:24:07.553077 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552869    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-certs\") pod \"2af13a3b-a68f-4156-9892-cb16249da39a\" (UID: \"2af13a3b-a68f-4156-9892-cb16249da39a\") "
Aug 02 11:24:07.553077 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552920    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2af13a3b-a68f-4156-9892-cb16249da39a" (UID: "2af13a3b-a68f-4156-9892-cb16249da39a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:24:07.553077 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.552947    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-data" (OuterVolumeSpecName: "istio-data") pod "2af13a3b-a68f-4156-9892-cb16249da39a" (UID: "2af13a3b-a68f-4156-9892-cb16249da39a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:24:07.553301 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.553210    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2af13a3b-a68f-4156-9892-cb16249da39a" (UID: "2af13a3b-a68f-4156-9892-cb16249da39a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:24:07.553301 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.553223    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-data\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:07.553301 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.553243    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:07.553301 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.553233    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2af13a3b-a68f-4156-9892-cb16249da39a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2af13a3b-a68f-4156-9892-cb16249da39a" (UID: "2af13a3b-a68f-4156-9892-cb16249da39a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:24:07.553531 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.553501    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2af13a3b-a68f-4156-9892-cb16249da39a" (UID: "2af13a3b-a68f-4156-9892-cb16249da39a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:24:07.555018 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.554994    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2af13a3b-a68f-4156-9892-cb16249da39a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2af13a3b-a68f-4156-9892-cb16249da39a" (UID: "2af13a3b-a68f-4156-9892-cb16249da39a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 02 11:24:07.555351 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.555324    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-kube-api-access-t2bh4" (OuterVolumeSpecName: "kube-api-access-t2bh4") pod "2af13a3b-a68f-4156-9892-cb16249da39a" (UID: "2af13a3b-a68f-4156-9892-cb16249da39a"). InnerVolumeSpecName "kube-api-access-t2bh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:24:07.555610 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.555582    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2af13a3b-a68f-4156-9892-cb16249da39a" (UID: "2af13a3b-a68f-4156-9892-cb16249da39a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:24:07.555695 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.555679    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-istio-token" (OuterVolumeSpecName: "istio-token") pod "2af13a3b-a68f-4156-9892-cb16249da39a" (UID: "2af13a3b-a68f-4156-9892-cb16249da39a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:24:07.654333 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.654301    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2bh4\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-kube-api-access-t2bh4\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:07.654333 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.654329    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af13a3b-a68f-4156-9892-cb16249da39a-istiod-ca-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:07.654500 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.654343    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af13a3b-a68f-4156-9892-cb16249da39a-istio-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:07.654500 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.654354    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-istio-envoy\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:07.654500 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.654364    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-credential-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:07.654500 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.654375    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af13a3b-a68f-4156-9892-cb16249da39a-workload-certs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:07.654500 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:07.654386    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af13a3b-a68f-4156-9892-cb16249da39a-istio-podinfo\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:08.038115 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.038084    2576 generic.go:356] "Generic (PLEG): container finished" podID="2af13a3b-a68f-4156-9892-cb16249da39a" containerID="8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7" exitCode=0
Aug 02 11:24:08.038297 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.038130    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"
Aug 02 11:24:08.038297 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.038141    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" event={"ID":"2af13a3b-a68f-4156-9892-cb16249da39a","Type":"ContainerDied","Data":"8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7"}
Aug 02 11:24:08.038297 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.038184    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh" event={"ID":"2af13a3b-a68f-4156-9892-cb16249da39a","Type":"ContainerDied","Data":"29b775b87bb2891965ddaf230fe9e16df76d51f876eb6f6fe8f5e3dd3a06d234"}
Aug 02 11:24:08.038297 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.038204    2576 scope.go:117] "RemoveContainer" containerID="8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7"
Aug 02 11:24:08.048424 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.048408    2576 scope.go:117] "RemoveContainer" containerID="e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba"
Aug 02 11:24:08.056767 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.056748    2576 scope.go:117] "RemoveContainer" containerID="8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7"
Aug 02 11:24:08.057007 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:24:08.056982    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7\": container with ID starting with 8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7 not found: ID does not exist" containerID="8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7"
Aug 02 11:24:08.057081 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.057021    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7"} err="failed to get container status \"8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7\": rpc error: code = NotFound desc = could not find container \"8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7\": container with ID starting with 8df3d19a8d8fd7af9fae3648dd71cab1329c5ec9292593da0b754163e0c5eca7 not found: ID does not exist"
Aug 02 11:24:08.057124 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.057089    2576 scope.go:117] "RemoveContainer" containerID="e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba"
Aug 02 11:24:08.057421 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:24:08.057394    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba\": container with ID starting with e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba not found: ID does not exist" containerID="e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba"
Aug 02 11:24:08.057513 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.057429    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba"} err="failed to get container status \"e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba\": rpc error: code = NotFound desc = could not find container \"e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba\": container with ID starting with e58e48ea1b9cb757a0f165567fed630a2641f2a2b760b4ef263eba5c0d38bfba not found: ID does not exist"
Aug 02 11:24:08.060107 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.060083    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"]
Aug 02 11:24:08.063360 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.063341    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8cfbc4-openshift-default-6d988578bd-5jtfh"]
Aug 02 11:24:08.303226 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.303134    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"]
Aug 02 11:24:08.303742 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.303682    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq" podUID="c899af30-26f0-4b2a-8453-0ba9df9a0991" containerName="maas-api" containerID="cri-o://01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d" gracePeriod=30
Aug 02 11:24:08.566212 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.566152    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:24:08.663726 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.663701    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c899af30-26f0-4b2a-8453-0ba9df9a0991-maas-api-tls\") pod \"c899af30-26f0-4b2a-8453-0ba9df9a0991\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") "
Aug 02 11:24:08.663886 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.663753    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c899af30-26f0-4b2a-8453-0ba9df9a0991-openshift-service-ca\") pod \"c899af30-26f0-4b2a-8453-0ba9df9a0991\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") "
Aug 02 11:24:08.663886 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.663793    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6rkr\" (UniqueName: \"kubernetes.io/projected/c899af30-26f0-4b2a-8453-0ba9df9a0991-kube-api-access-v6rkr\") pod \"c899af30-26f0-4b2a-8453-0ba9df9a0991\" (UID: \"c899af30-26f0-4b2a-8453-0ba9df9a0991\") "
Aug 02 11:24:08.664203 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.664167    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c899af30-26f0-4b2a-8453-0ba9df9a0991-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c899af30-26f0-4b2a-8453-0ba9df9a0991" (UID: "c899af30-26f0-4b2a-8453-0ba9df9a0991"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:24:08.666071 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.666049    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c899af30-26f0-4b2a-8453-0ba9df9a0991-kube-api-access-v6rkr" (OuterVolumeSpecName: "kube-api-access-v6rkr") pod "c899af30-26f0-4b2a-8453-0ba9df9a0991" (UID: "c899af30-26f0-4b2a-8453-0ba9df9a0991"). InnerVolumeSpecName "kube-api-access-v6rkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:24:08.666149 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.666091    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c899af30-26f0-4b2a-8453-0ba9df9a0991-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c899af30-26f0-4b2a-8453-0ba9df9a0991" (UID: "c899af30-26f0-4b2a-8453-0ba9df9a0991"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:24:08.764986 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.764949    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c899af30-26f0-4b2a-8453-0ba9df9a0991-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:08.764986 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.764979    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c899af30-26f0-4b2a-8453-0ba9df9a0991-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:08.764986 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.764991    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6rkr\" (UniqueName: \"kubernetes.io/projected/c899af30-26f0-4b2a-8453-0ba9df9a0991-kube-api-access-v6rkr\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:24:08.951709 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:08.951675    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" path="/var/lib/kubelet/pods/2af13a3b-a68f-4156-9892-cb16249da39a/volumes"
Aug 02 11:24:09.046627 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:09.046591    2576 generic.go:356] "Generic (PLEG): container finished" podID="c899af30-26f0-4b2a-8453-0ba9df9a0991" containerID="01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d" exitCode=0
Aug 02 11:24:09.046790 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:09.046642    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"
Aug 02 11:24:09.046790 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:09.046682    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq" event={"ID":"c899af30-26f0-4b2a-8453-0ba9df9a0991","Type":"ContainerDied","Data":"01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d"}
Aug 02 11:24:09.046790 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:09.046734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq" event={"ID":"c899af30-26f0-4b2a-8453-0ba9df9a0991","Type":"ContainerDied","Data":"97f84aafc61c768f4dceb3baa90d502277caf709db851c697b8243fd848d000b"}
Aug 02 11:24:09.046790 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:09.046755    2576 scope.go:117] "RemoveContainer" containerID="01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d"
Aug 02 11:24:09.055695 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:09.055680    2576 scope.go:117] "RemoveContainer" containerID="01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d"
Aug 02 11:24:09.055947 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:24:09.055929    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d\": container with ID starting with 01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d not found: ID does not exist" containerID="01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d"
Aug 02 11:24:09.055990 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:09.055955    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d"} err="failed to get container status \"01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d\": rpc error: code = NotFound desc = could not find container \"01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d\": container with ID starting with 01abcb63de70830b2bfc1000aee3e5bd10600bb923326ff8c9244432432c401d not found: ID does not exist"
Aug 02 11:24:09.063254 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:09.063233    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"]
Aug 02 11:24:09.066921 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:09.066902    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5afe3e-54f79d9dd-sgqrq"]
Aug 02 11:24:10.950682 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:10.950593    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c899af30-26f0-4b2a-8453-0ba9df9a0991" path="/var/lib/kubelet/pods/c899af30-26f0-4b2a-8453-0ba9df9a0991/volumes"
Aug 02 11:24:16.229044 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.228999    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"]
Aug 02 11:24:16.229534 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229518    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy"
Aug 02 11:24:16.229592 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229537    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy"
Aug 02 11:24:16.229592 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229553    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c899af30-26f0-4b2a-8453-0ba9df9a0991" containerName="maas-api"
Aug 02 11:24:16.229592 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229558    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c899af30-26f0-4b2a-8453-0ba9df9a0991" containerName="maas-api"
Aug 02 11:24:16.229592 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229569    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4e6bbe6-5e69-496d-abfa-36930e82ca98" containerName="maas-api"
Aug 02 11:24:16.229592 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229575    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4e6bbe6-5e69-496d-abfa-36930e82ca98" containerName="maas-api"
Aug 02 11:24:16.229745 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229641    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy"
Aug 02 11:24:16.229745 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229648    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy"
Aug 02 11:24:16.229745 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229656    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c899af30-26f0-4b2a-8453-0ba9df9a0991" containerName="maas-api"
Aug 02 11:24:16.229745 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.229662    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4e6bbe6-5e69-496d-abfa-36930e82ca98" containerName="maas-api"
Aug 02 11:24:16.234182 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.234160    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.236792 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.236772    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-3951f6-openshift-default-dockercfg-9wlkv\""
Aug 02 11:24:16.244117 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.244092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"]
Aug 02 11:24:16.332100 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.332070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-token\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.332257 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.332110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-data\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.332257 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.332132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istiod-ca-cert\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.332257 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.332238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-socket\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.332361 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.332293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-certs\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.332361 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.332353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-credential-socket\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.332430 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.332378    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-envoy\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.332464 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.332445    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42jmw\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-kube-api-access-42jmw\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.332503 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.332470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-podinfo\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433075 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-token\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433075 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-data\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433318 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istiod-ca-cert\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433318 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-socket\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433426 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433340    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-certs\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433426 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-credential-socket\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433540 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433423    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-envoy\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433540 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433433    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-data\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433540 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42jmw\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-kube-api-access-42jmw\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433540 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-podinfo\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433754 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433617    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-socket\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433754 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-certs\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433754 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433732    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-credential-socket\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.433872 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.433826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istiod-ca-cert\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.435965 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.435935    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-envoy\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.436141 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.436120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-podinfo\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.441202 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.441180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42jmw\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-kube-api-access-42jmw\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.441316 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.441241    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-token\") pod \"e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") " pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.546197 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.546113    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:16.689075 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.689051    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"]
Aug 02 11:24:16.691188 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:24:16.691156    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e75a9e3_f08c_48a1_90e6_621cdcc99655.slice/crio-6417ad03bb6544be9d6ce3332e5d1ac8379d91f3fe38156d97efbd116dd8eca1 WatchSource:0}: Error finding container 6417ad03bb6544be9d6ce3332e5d1ac8379d91f3fe38156d97efbd116dd8eca1: Status 404 returned error can't find the container with id 6417ad03bb6544be9d6ce3332e5d1ac8379d91f3fe38156d97efbd116dd8eca1
Aug 02 11:24:16.693658 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.693622    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:24:16.693779 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.693709    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:24:16.693779 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:16.693746    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:24:17.103530 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:17.103489    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" event={"ID":"8e75a9e3-f08c-48a1-90e6-621cdcc99655","Type":"ContainerStarted","Data":"12db0ccd990ae2ad2e24696835b036714a0c47e6e3c6b5f813a706cf1cf66964"}
Aug 02 11:24:17.103742 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:17.103583    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" event={"ID":"8e75a9e3-f08c-48a1-90e6-621cdcc99655","Type":"ContainerStarted","Data":"6417ad03bb6544be9d6ce3332e5d1ac8379d91f3fe38156d97efbd116dd8eca1"}
Aug 02 11:24:17.122470 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:17.122416    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" podStartSLOduration=1.1223998609999999 podStartE2EDuration="1.122399861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:24:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:24:17.121849355 +0000 UTC m=+3058.652431989" watchObservedRunningTime="2026-08-02 11:24:17.122399861 +0000 UTC m=+3058.652982490"
Aug 02 11:24:17.546757 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:17.546723    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:17.551764 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:17.551743    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:18.110497 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:18.110464    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:18.111522 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:18.111501    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:20.048985 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.048941    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"]
Aug 02 11:24:20.049563 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.049542    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy"
Aug 02 11:24:20.049660 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.049566    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af13a3b-a68f-4156-9892-cb16249da39a" containerName="istio-proxy"
Aug 02 11:24:20.053893 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.053872    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.056375 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.056352    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-3951f6-serving-cert\""
Aug 02 11:24:20.067535 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.064104    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"]
Aug 02 11:24:20.168697 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.168663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph8hb\" (UniqueName: \"kubernetes.io/projected/f2d1e3b6-d262-46a6-b834-e6288230e9b4-kube-api-access-ph8hb\") pod \"maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.168881 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.168810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d1e3b6-d262-46a6-b834-e6288230e9b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.168881 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.168836    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d1e3b6-d262-46a6-b834-e6288230e9b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.270104 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.270071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d1e3b6-d262-46a6-b834-e6288230e9b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.270104 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.270107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d1e3b6-d262-46a6-b834-e6288230e9b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.270311 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.270169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph8hb\" (UniqueName: \"kubernetes.io/projected/f2d1e3b6-d262-46a6-b834-e6288230e9b4-kube-api-access-ph8hb\") pod \"maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.270835 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.270756    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d1e3b6-d262-46a6-b834-e6288230e9b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.272690 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.272665    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d1e3b6-d262-46a6-b834-e6288230e9b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.279319 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.279292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph8hb\" (UniqueName: \"kubernetes.io/projected/f2d1e3b6-d262-46a6-b834-e6288230e9b4-kube-api-access-ph8hb\") pod \"maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.369431 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.369373    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:20.522022 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:20.521850    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"]
Aug 02 11:24:20.523329 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:24:20.523290    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d1e3b6_d262_46a6_b834_e6288230e9b4.slice/crio-1bf95abef1d6b7d3602746fec6a6ead015982d75d0d0bd3fdbb830764ebe4461 WatchSource:0}: Error finding container 1bf95abef1d6b7d3602746fec6a6ead015982d75d0d0bd3fdbb830764ebe4461: Status 404 returned error can't find the container with id 1bf95abef1d6b7d3602746fec6a6ead015982d75d0d0bd3fdbb830764ebe4461
Aug 02 11:24:21.129084 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:21.129021    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx" event={"ID":"f2d1e3b6-d262-46a6-b834-e6288230e9b4","Type":"ContainerStarted","Data":"c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c"}
Aug 02 11:24:21.129459 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:21.129103    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx" event={"ID":"f2d1e3b6-d262-46a6-b834-e6288230e9b4","Type":"ContainerStarted","Data":"1bf95abef1d6b7d3602746fec6a6ead015982d75d0d0bd3fdbb830764ebe4461"}
Aug 02 11:24:21.129459 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:21.129134    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:21.146305 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:21.146256    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx" podStartSLOduration=0.729672943 podStartE2EDuration="1.146240953s" totalImagesPullingTime="416.56801ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:24:21.144454312 +0000 UTC m=+3062.675036956" watchObservedRunningTime="2026-08-02 11:24:21.146240953 +0000 UTC m=+3062.676823583"
Aug 02 11:24:27.141276 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:27.141249    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:24:33.112516 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:33.112480    2576 patch_prober.go:28] interesting pod/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Aug 02 11:24:33.112943 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:33.112567    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Aug 02 11:24:33.199228 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:33.199191    2576 generic.go:356] "Generic (PLEG): container finished" podID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerID="12db0ccd990ae2ad2e24696835b036714a0c47e6e3c6b5f813a706cf1cf66964" exitCode=0
Aug 02 11:24:33.199408 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:33.199242    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" event={"ID":"8e75a9e3-f08c-48a1-90e6-621cdcc99655","Type":"ContainerDied","Data":"12db0ccd990ae2ad2e24696835b036714a0c47e6e3c6b5f813a706cf1cf66964"}
Aug 02 11:24:33.199670 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:33.199653    2576 scope.go:117] "RemoveContainer" containerID="12db0ccd990ae2ad2e24696835b036714a0c47e6e3c6b5f813a706cf1cf66964"
Aug 02 11:24:33.200699 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:33.200670    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:24:33.200770 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:33.200755    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:24:33.200809 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:33.200792    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:24:34.218573 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:34.218537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" event={"ID":"8e75a9e3-f08c-48a1-90e6-621cdcc99655","Type":"ContainerStarted","Data":"f13ce6b7a81d3f6da7672528eac8efaa9ab9f9e5268dacca7c920ae698eafa85"}
Aug 02 11:24:34.547249 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:34.547160    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:34.547428 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:34.547405    2576 patch_prober.go:28] interesting pod/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Aug 02 11:24:34.547472 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:34.547455    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Aug 02 11:24:35.547292 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:35.547254    2576 patch_prober.go:28] interesting pod/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Aug 02 11:24:35.547677 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:35.547315    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Aug 02 11:24:36.547429 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:36.547394    2576 patch_prober.go:28] interesting pod/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Aug 02 11:24:36.547795 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:36.547449    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Aug 02 11:24:37.547221 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:37.547189    2576 patch_prober.go:28] interesting pod/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Aug 02 11:24:37.547400 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:37.547241    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Aug 02 11:24:38.547386 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:38.547347    2576 patch_prober.go:28] interesting pod/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Aug 02 11:24:38.547831 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:38.547402    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Aug 02 11:24:39.550431 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:39.550401    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:39.550813 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:39.550793    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:39.551486 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:39.551468    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:24:48.852694 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:48.852663    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"]
Aug 02 11:24:48.856762 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:48.856741    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:48.859320 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:48.859298    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-4f1bb7-serving-cert\""
Aug 02 11:24:48.865590 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:48.865567    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"]
Aug 02 11:24:49.038814 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.038777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b84kx\" (UniqueName: \"kubernetes.io/projected/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-kube-api-access-b84kx\") pod \"maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.038993 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.038828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.038993 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.038918    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-maas-api-tls\") pod \"maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.139552 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.139465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-maas-api-tls\") pod \"maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.139552 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.139548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b84kx\" (UniqueName: \"kubernetes.io/projected/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-kube-api-access-b84kx\") pod \"maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.139790 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.139595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.140284 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.140259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.142119 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.142101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-maas-api-tls\") pod \"maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.156747 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.156724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b84kx\" (UniqueName: \"kubernetes.io/projected/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-kube-api-access-b84kx\") pod \"maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.171480 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.171460    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:49.316294 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.316265    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"]
Aug 02 11:24:49.317911 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:24:49.317885    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b9a0ec_da4d_4d23_b8ce_a4fa12f97659.slice/crio-4dc69b0b1bc855c9c950b5992cc1b167aaacc7cc6d2d6dc51665922007cad0cb WatchSource:0}: Error finding container 4dc69b0b1bc855c9c950b5992cc1b167aaacc7cc6d2d6dc51665922007cad0cb: Status 404 returned error can't find the container with id 4dc69b0b1bc855c9c950b5992cc1b167aaacc7cc6d2d6dc51665922007cad0cb
Aug 02 11:24:49.319803 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:49.319785    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 02 11:24:50.310492 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:50.310451    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr" event={"ID":"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659","Type":"ContainerStarted","Data":"c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de"}
Aug 02 11:24:50.310864 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:50.310512    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:24:50.310864 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:50.310531    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr" event={"ID":"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659","Type":"ContainerStarted","Data":"4dc69b0b1bc855c9c950b5992cc1b167aaacc7cc6d2d6dc51665922007cad0cb"}
Aug 02 11:24:50.327973 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:50.327927    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr" podStartSLOduration=1.91639498 podStartE2EDuration="2.327909504s" totalImagesPullingTime="411.514524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:24:50.325687611 +0000 UTC m=+3091.856270242" watchObservedRunningTime="2026-08-02 11:24:50.327909504 +0000 UTC m=+3091.858492132"
Aug 02 11:24:56.322349 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:24:56.322315    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:28:19.204512 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:28:19.204487    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:28:19.226729 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:28:19.226704    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:29:19.362419 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:29:19.362378    2576 scope.go:117] "RemoveContainer" containerID="985ed7a1438f3417930adddd3800dc761ecf44676dc6b1df5328cc91b1e7b5c7"
Aug 02 11:33:19.241237 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:19.241209    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:33:19.271502 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:19.271469    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:33:31.753241 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:31.753204    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7333-139910175261744"]
Aug 02 11:33:31.756995 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:31.756968    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175261744"
Aug 02 11:33:31.759390 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:31.759369    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mqz2w\""
Aug 02 11:33:31.760362 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:31.760335    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175261744"]
Aug 02 11:33:31.863518 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:31.863489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt5j6\" (UniqueName: \"kubernetes.io/projected/bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c-kube-api-access-dt5j6\") pod \"test-curl-7333-139910175261744\" (UID: \"bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c\") " pod="opendatahub/test-curl-7333-139910175261744"
Aug 02 11:33:31.964900 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:31.964869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5j6\" (UniqueName: \"kubernetes.io/projected/bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c-kube-api-access-dt5j6\") pod \"test-curl-7333-139910175261744\" (UID: \"bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c\") " pod="opendatahub/test-curl-7333-139910175261744"
Aug 02 11:33:31.973102 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:31.973080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt5j6\" (UniqueName: \"kubernetes.io/projected/bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c-kube-api-access-dt5j6\") pod \"test-curl-7333-139910175261744\" (UID: \"bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c\") " pod="opendatahub/test-curl-7333-139910175261744"
Aug 02 11:33:32.067947 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:32.067853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175261744"
Aug 02 11:33:32.205536 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:32.205494    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175261744"]
Aug 02 11:33:32.207060 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:33:32.207006    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdf90a7b_d36e_46ea_a2d9_8195c1c0ba3c.slice/crio-cfa54016ea55016581ad589a915f5492c2f7d70a4a299dc2f006323be9ad69e3 WatchSource:0}: Error finding container cfa54016ea55016581ad589a915f5492c2f7d70a4a299dc2f006323be9ad69e3: Status 404 returned error can't find the container with id cfa54016ea55016581ad589a915f5492c2f7d70a4a299dc2f006323be9ad69e3
Aug 02 11:33:32.208745 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:32.208725    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 02 11:33:32.316099 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:32.316069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175261744" event={"ID":"bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c","Type":"ContainerStarted","Data":"cfa54016ea55016581ad589a915f5492c2f7d70a4a299dc2f006323be9ad69e3"}
Aug 02 11:33:33.326291 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:33.326255    2576 generic.go:356] "Generic (PLEG): container finished" podID="bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c" containerID="8332e99219fac78213f817a6562401d0dc03574c0f51f65b02ab2b5e0e4c2033" exitCode=0
Aug 02 11:33:33.326675 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:33.326336    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175261744" event={"ID":"bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c","Type":"ContainerDied","Data":"8332e99219fac78213f817a6562401d0dc03574c0f51f65b02ab2b5e0e4c2033"}
Aug 02 11:33:34.477165 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.477140    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175261744"
Aug 02 11:33:34.511746 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.511715    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7333-139910175261744_bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c/test-curl-7333-139910175261744/0.log"
Aug 02 11:33:34.531677 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.531649    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7333-139910175261744"]
Aug 02 11:33:34.536679 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.536657    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7333-139910175261744"]
Aug 02 11:33:34.591893 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.591823    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt5j6\" (UniqueName: \"kubernetes.io/projected/bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c-kube-api-access-dt5j6\") pod \"bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c\" (UID: \"bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c\") "
Aug 02 11:33:34.594209 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.594176    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c-kube-api-access-dt5j6" (OuterVolumeSpecName: "kube-api-access-dt5j6") pod "bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c" (UID: "bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c"). InnerVolumeSpecName "kube-api-access-dt5j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:33:34.653018 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.652980    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7333-139910175225008"]
Aug 02 11:33:34.653479 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.653460    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c" containerName="test-curl-7333-139910175261744"
Aug 02 11:33:34.653569 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.653481    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c" containerName="test-curl-7333-139910175261744"
Aug 02 11:33:34.653624 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.653582    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c" containerName="test-curl-7333-139910175261744"
Aug 02 11:33:34.656724 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.656706    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175225008"
Aug 02 11:33:34.662078 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.662055    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175225008"]
Aug 02 11:33:34.693468 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.693441    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt5j6\" (UniqueName: \"kubernetes.io/projected/bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c-kube-api-access-dt5j6\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:33:34.795004 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.794969    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpcqv\" (UniqueName: \"kubernetes.io/projected/ecbde327-4f78-4a1d-80d9-70316b2ce570-kube-api-access-rpcqv\") pod \"test-curl-7333-139910175225008\" (UID: \"ecbde327-4f78-4a1d-80d9-70316b2ce570\") " pod="opendatahub/test-curl-7333-139910175225008"
Aug 02 11:33:34.896209 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.896127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpcqv\" (UniqueName: \"kubernetes.io/projected/ecbde327-4f78-4a1d-80d9-70316b2ce570-kube-api-access-rpcqv\") pod \"test-curl-7333-139910175225008\" (UID: \"ecbde327-4f78-4a1d-80d9-70316b2ce570\") " pod="opendatahub/test-curl-7333-139910175225008"
Aug 02 11:33:34.904041 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.903994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpcqv\" (UniqueName: \"kubernetes.io/projected/ecbde327-4f78-4a1d-80d9-70316b2ce570-kube-api-access-rpcqv\") pod \"test-curl-7333-139910175225008\" (UID: \"ecbde327-4f78-4a1d-80d9-70316b2ce570\") " pod="opendatahub/test-curl-7333-139910175225008"
Aug 02 11:33:34.951258 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.951229    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c" path="/var/lib/kubelet/pods/bdf90a7b-d36e-46ea-a2d9-8195c1c0ba3c/volumes"
Aug 02 11:33:34.967566 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:34.967544    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175225008"
Aug 02 11:33:35.105503 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:35.105479    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175225008"]
Aug 02 11:33:35.108065 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:33:35.108017    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecbde327_4f78_4a1d_80d9_70316b2ce570.slice/crio-249d5cc88e417f7c3acb3c66018577e76750d66dce51ba9caffcb0e679ad69df WatchSource:0}: Error finding container 249d5cc88e417f7c3acb3c66018577e76750d66dce51ba9caffcb0e679ad69df: Status 404 returned error can't find the container with id 249d5cc88e417f7c3acb3c66018577e76750d66dce51ba9caffcb0e679ad69df
Aug 02 11:33:35.339971 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:35.339936    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175261744"
Aug 02 11:33:35.340183 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:35.339987    2576 scope.go:117] "RemoveContainer" containerID="8332e99219fac78213f817a6562401d0dc03574c0f51f65b02ab2b5e0e4c2033"
Aug 02 11:33:35.341424 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:35.341401    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175225008" event={"ID":"ecbde327-4f78-4a1d-80d9-70316b2ce570","Type":"ContainerStarted","Data":"249d5cc88e417f7c3acb3c66018577e76750d66dce51ba9caffcb0e679ad69df"}
Aug 02 11:33:36.350213 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:36.350181    2576 generic.go:356] "Generic (PLEG): container finished" podID="ecbde327-4f78-4a1d-80d9-70316b2ce570" containerID="500c128d866ce233ba5a9ae88044d689d4f8af1af1707d1d064ec015ae80dca2" exitCode=0
Aug 02 11:33:36.350592 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:36.350238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175225008" event={"ID":"ecbde327-4f78-4a1d-80d9-70316b2ce570","Type":"ContainerDied","Data":"500c128d866ce233ba5a9ae88044d689d4f8af1af1707d1d064ec015ae80dca2"}
Aug 02 11:33:37.511426 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.511400    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175225008"
Aug 02 11:33:37.530078 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.530047    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7333-139910175225008_ecbde327-4f78-4a1d-80d9-70316b2ce570/test-curl-7333-139910175225008/0.log"
Aug 02 11:33:37.550296 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.550229    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7333-139910175225008"]
Aug 02 11:33:37.553357 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.553335    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7333-139910175225008"]
Aug 02 11:33:37.619616 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.619526    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpcqv\" (UniqueName: \"kubernetes.io/projected/ecbde327-4f78-4a1d-80d9-70316b2ce570-kube-api-access-rpcqv\") pod \"ecbde327-4f78-4a1d-80d9-70316b2ce570\" (UID: \"ecbde327-4f78-4a1d-80d9-70316b2ce570\") "
Aug 02 11:33:37.621742 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.621718    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecbde327-4f78-4a1d-80d9-70316b2ce570-kube-api-access-rpcqv" (OuterVolumeSpecName: "kube-api-access-rpcqv") pod "ecbde327-4f78-4a1d-80d9-70316b2ce570" (UID: "ecbde327-4f78-4a1d-80d9-70316b2ce570"). InnerVolumeSpecName "kube-api-access-rpcqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:33:37.712565 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.712531    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:37.712981 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.712967    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecbde327-4f78-4a1d-80d9-70316b2ce570" containerName="test-curl-7333-139910175225008"
Aug 02 11:33:37.713042 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.712983    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecbde327-4f78-4a1d-80d9-70316b2ce570" containerName="test-curl-7333-139910175225008"
Aug 02 11:33:37.713096 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.713085    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecbde327-4f78-4a1d-80d9-70316b2ce570" containerName="test-curl-7333-139910175225008"
Aug 02 11:33:37.716347 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.716329    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:37.719710 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.719565    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:37.720464 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.720438    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpcqv\" (UniqueName: \"kubernetes.io/projected/ecbde327-4f78-4a1d-80d9-70316b2ce570-kube-api-access-rpcqv\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:33:37.820914 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.820875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhhjf\" (UniqueName: \"kubernetes.io/projected/c93c39d5-ee16-4f81-8810-348d2e58a0e7-kube-api-access-lhhjf\") pod \"test-curl-7333-139910175027680\" (UID: \"c93c39d5-ee16-4f81-8810-348d2e58a0e7\") " pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:37.922394 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.922346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhhjf\" (UniqueName: \"kubernetes.io/projected/c93c39d5-ee16-4f81-8810-348d2e58a0e7-kube-api-access-lhhjf\") pod \"test-curl-7333-139910175027680\" (UID: \"c93c39d5-ee16-4f81-8810-348d2e58a0e7\") " pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:37.930112 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:37.930087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhhjf\" (UniqueName: \"kubernetes.io/projected/c93c39d5-ee16-4f81-8810-348d2e58a0e7-kube-api-access-lhhjf\") pod \"test-curl-7333-139910175027680\" (UID: \"c93c39d5-ee16-4f81-8810-348d2e58a0e7\") " pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:38.026216 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:38.026185    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:38.167150 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:38.167125    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:38.169012 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:33:38.168987    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc93c39d5_ee16_4f81_8810_348d2e58a0e7.slice/crio-201d3e1aa892a7e445a34d3674a1bc05a7a47f48e8625ae508f5701b23d93d62 WatchSource:0}: Error finding container 201d3e1aa892a7e445a34d3674a1bc05a7a47f48e8625ae508f5701b23d93d62: Status 404 returned error can't find the container with id 201d3e1aa892a7e445a34d3674a1bc05a7a47f48e8625ae508f5701b23d93d62
Aug 02 11:33:38.363950 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:38.363916    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175225008"
Aug 02 11:33:38.363950 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:38.363952    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="249d5cc88e417f7c3acb3c66018577e76750d66dce51ba9caffcb0e679ad69df"
Aug 02 11:33:38.365112 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:38.365088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175027680" event={"ID":"c93c39d5-ee16-4f81-8810-348d2e58a0e7","Type":"ContainerStarted","Data":"201d3e1aa892a7e445a34d3674a1bc05a7a47f48e8625ae508f5701b23d93d62"}
Aug 02 11:33:38.374661 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:33:38.374635    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7333-139910175225008\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="ecbde327-4f78-4a1d-80d9-70316b2ce570" pod="opendatahub/test-curl-7333-139910175225008"
Aug 02 11:33:38.949382 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:33:38.949332    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7333-139910175225008\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="ecbde327-4f78-4a1d-80d9-70316b2ce570" pod="opendatahub/test-curl-7333-139910175225008"
Aug 02 11:33:38.950599 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:38.950579    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecbde327-4f78-4a1d-80d9-70316b2ce570" path="/var/lib/kubelet/pods/ecbde327-4f78-4a1d-80d9-70316b2ce570/volumes"
Aug 02 11:33:39.372911 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:39.372815    2576 generic.go:356] "Generic (PLEG): container finished" podID="c93c39d5-ee16-4f81-8810-348d2e58a0e7" containerID="859373d799f70837d83fdfcaef275a55e318e0981fb9c56d9b056308cf5e8d33" exitCode=0
Aug 02 11:33:39.372911 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:39.372876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175027680" event={"ID":"c93c39d5-ee16-4f81-8810-348d2e58a0e7","Type":"ContainerDied","Data":"859373d799f70837d83fdfcaef275a55e318e0981fb9c56d9b056308cf5e8d33"}
Aug 02 11:33:40.522398 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.522377    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:40.541372 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.541343    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7333-139910175027680_c93c39d5-ee16-4f81-8810-348d2e58a0e7/test-curl-7333-139910175027680/0.log"
Aug 02 11:33:40.563321 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.563286    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:40.565613 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.565588    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:40.648668 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.648643    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhhjf\" (UniqueName: \"kubernetes.io/projected/c93c39d5-ee16-4f81-8810-348d2e58a0e7-kube-api-access-lhhjf\") pod \"c93c39d5-ee16-4f81-8810-348d2e58a0e7\" (UID: \"c93c39d5-ee16-4f81-8810-348d2e58a0e7\") "
Aug 02 11:33:40.650838 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.650813    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c93c39d5-ee16-4f81-8810-348d2e58a0e7-kube-api-access-lhhjf" (OuterVolumeSpecName: "kube-api-access-lhhjf") pod "c93c39d5-ee16-4f81-8810-348d2e58a0e7" (UID: "c93c39d5-ee16-4f81-8810-348d2e58a0e7"). InnerVolumeSpecName "kube-api-access-lhhjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:33:40.673182 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.673153    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:40.673567 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.673555    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93c39d5-ee16-4f81-8810-348d2e58a0e7" containerName="test-curl-7333-139910175027680"
Aug 02 11:33:40.673610 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.673570    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93c39d5-ee16-4f81-8810-348d2e58a0e7" containerName="test-curl-7333-139910175027680"
Aug 02 11:33:40.673654 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.673645    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93c39d5-ee16-4f81-8810-348d2e58a0e7" containerName="test-curl-7333-139910175027680"
Aug 02 11:33:40.676601 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.676586    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:40.682217 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.682192    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:40.757208 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.757172    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhhjf\" (UniqueName: \"kubernetes.io/projected/c93c39d5-ee16-4f81-8810-348d2e58a0e7-kube-api-access-lhhjf\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:33:40.858352 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.858313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cp6w\" (UniqueName: \"kubernetes.io/projected/741f19b3-66bd-43c2-bdee-a9197aedfde9-kube-api-access-7cp6w\") pod \"test-curl-7333-139910175027680\" (UID: \"741f19b3-66bd-43c2-bdee-a9197aedfde9\") " pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:40.952064 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.951950    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c93c39d5-ee16-4f81-8810-348d2e58a0e7" path="/var/lib/kubelet/pods/c93c39d5-ee16-4f81-8810-348d2e58a0e7/volumes"
Aug 02 11:33:40.959254 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.959221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cp6w\" (UniqueName: \"kubernetes.io/projected/741f19b3-66bd-43c2-bdee-a9197aedfde9-kube-api-access-7cp6w\") pod \"test-curl-7333-139910175027680\" (UID: \"741f19b3-66bd-43c2-bdee-a9197aedfde9\") " pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:40.967145 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.967119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cp6w\" (UniqueName: \"kubernetes.io/projected/741f19b3-66bd-43c2-bdee-a9197aedfde9-kube-api-access-7cp6w\") pod \"test-curl-7333-139910175027680\" (UID: \"741f19b3-66bd-43c2-bdee-a9197aedfde9\") " pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:40.988730 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:40.988701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:41.125877 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:41.125849    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:41.127245 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:33:41.127220    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod741f19b3_66bd_43c2_bdee_a9197aedfde9.slice/crio-0aa45aabe253c91f146c110cc7424aa87c3cafc4b6fa0eb10f056ca4b130c993 WatchSource:0}: Error finding container 0aa45aabe253c91f146c110cc7424aa87c3cafc4b6fa0eb10f056ca4b130c993: Status 404 returned error can't find the container with id 0aa45aabe253c91f146c110cc7424aa87c3cafc4b6fa0eb10f056ca4b130c993
Aug 02 11:33:41.385557 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:41.385532    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175027680" event={"ID":"741f19b3-66bd-43c2-bdee-a9197aedfde9","Type":"ContainerStarted","Data":"0aa45aabe253c91f146c110cc7424aa87c3cafc4b6fa0eb10f056ca4b130c993"}
Aug 02 11:33:41.386734 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:41.386716    2576 scope.go:117] "RemoveContainer" containerID="859373d799f70837d83fdfcaef275a55e318e0981fb9c56d9b056308cf5e8d33"
Aug 02 11:33:41.386822 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:41.386749    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:42.394891 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:42.394854    2576 generic.go:356] "Generic (PLEG): container finished" podID="741f19b3-66bd-43c2-bdee-a9197aedfde9" containerID="bc7a1e2c277c0de31501c3dd63004795c6ef470e47f18cd640ba6cfc86cc5007" exitCode=0
Aug 02 11:33:42.395302 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:42.394935    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175027680" event={"ID":"741f19b3-66bd-43c2-bdee-a9197aedfde9","Type":"ContainerDied","Data":"bc7a1e2c277c0de31501c3dd63004795c6ef470e47f18cd640ba6cfc86cc5007"}
Aug 02 11:33:43.540732 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.540706    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:43.558862 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.558835    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7333-139910175027680_741f19b3-66bd-43c2-bdee-a9197aedfde9/test-curl-7333-139910175027680/0.log"
Aug 02 11:33:43.581809 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.581779    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:43.584135 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.584113    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7333-139910175027680"]
Aug 02 11:33:43.686977 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.686887    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cp6w\" (UniqueName: \"kubernetes.io/projected/741f19b3-66bd-43c2-bdee-a9197aedfde9-kube-api-access-7cp6w\") pod \"741f19b3-66bd-43c2-bdee-a9197aedfde9\" (UID: \"741f19b3-66bd-43c2-bdee-a9197aedfde9\") "
Aug 02 11:33:43.689160 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.689135    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/741f19b3-66bd-43c2-bdee-a9197aedfde9-kube-api-access-7cp6w" (OuterVolumeSpecName: "kube-api-access-7cp6w") pod "741f19b3-66bd-43c2-bdee-a9197aedfde9" (UID: "741f19b3-66bd-43c2-bdee-a9197aedfde9"). InnerVolumeSpecName "kube-api-access-7cp6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:33:43.703305 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.703276    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7333-139910175028688"]
Aug 02 11:33:43.703744 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.703730    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="741f19b3-66bd-43c2-bdee-a9197aedfde9" containerName="test-curl-7333-139910175027680"
Aug 02 11:33:43.703797 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.703747    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="741f19b3-66bd-43c2-bdee-a9197aedfde9" containerName="test-curl-7333-139910175027680"
Aug 02 11:33:43.703845 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.703808    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="741f19b3-66bd-43c2-bdee-a9197aedfde9" containerName="test-curl-7333-139910175027680"
Aug 02 11:33:43.706752 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.706737    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175028688"
Aug 02 11:33:43.712812 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.712790    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175028688"]
Aug 02 11:33:43.787812 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.787773    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cp6w\" (UniqueName: \"kubernetes.io/projected/741f19b3-66bd-43c2-bdee-a9197aedfde9-kube-api-access-7cp6w\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:33:43.888572 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.888529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhvg9\" (UniqueName: \"kubernetes.io/projected/d3798411-d5e9-4818-bd39-6ce8743d5b1e-kube-api-access-fhvg9\") pod \"test-curl-7333-139910175028688\" (UID: \"d3798411-d5e9-4818-bd39-6ce8743d5b1e\") " pod="opendatahub/test-curl-7333-139910175028688"
Aug 02 11:33:43.989999 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.989907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhvg9\" (UniqueName: \"kubernetes.io/projected/d3798411-d5e9-4818-bd39-6ce8743d5b1e-kube-api-access-fhvg9\") pod \"test-curl-7333-139910175028688\" (UID: \"d3798411-d5e9-4818-bd39-6ce8743d5b1e\") " pod="opendatahub/test-curl-7333-139910175028688"
Aug 02 11:33:43.998105 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:43.998077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhvg9\" (UniqueName: \"kubernetes.io/projected/d3798411-d5e9-4818-bd39-6ce8743d5b1e-kube-api-access-fhvg9\") pod \"test-curl-7333-139910175028688\" (UID: \"d3798411-d5e9-4818-bd39-6ce8743d5b1e\") " pod="opendatahub/test-curl-7333-139910175028688"
Aug 02 11:33:44.017956 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:44.017930    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175028688"
Aug 02 11:33:44.154793 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:44.154767    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7333-139910175028688"]
Aug 02 11:33:44.156232 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:33:44.156207    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3798411_d5e9_4818_bd39_6ce8743d5b1e.slice/crio-9f9725fedccfbb2c1cc00e827cd34b8a56e556b4aaa787ef4ee356cda130dc3a WatchSource:0}: Error finding container 9f9725fedccfbb2c1cc00e827cd34b8a56e556b4aaa787ef4ee356cda130dc3a: Status 404 returned error can't find the container with id 9f9725fedccfbb2c1cc00e827cd34b8a56e556b4aaa787ef4ee356cda130dc3a
Aug 02 11:33:44.407877 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:44.407848    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:44.408089 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:44.407890    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0aa45aabe253c91f146c110cc7424aa87c3cafc4b6fa0eb10f056ca4b130c993"
Aug 02 11:33:44.409050 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:44.409009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175028688" event={"ID":"d3798411-d5e9-4818-bd39-6ce8743d5b1e","Type":"ContainerStarted","Data":"9f9725fedccfbb2c1cc00e827cd34b8a56e556b4aaa787ef4ee356cda130dc3a"}
Aug 02 11:33:44.418483 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:33:44.418457    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7333-139910175027680\" is forbidden: User \"system:node:ip-10-0-135-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-193.ec2.internal' and this object" podUID="741f19b3-66bd-43c2-bdee-a9197aedfde9" pod="opendatahub/test-curl-7333-139910175027680"
Aug 02 11:33:44.951264 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:44.951232    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="741f19b3-66bd-43c2-bdee-a9197aedfde9" path="/var/lib/kubelet/pods/741f19b3-66bd-43c2-bdee-a9197aedfde9/volumes"
Aug 02 11:33:45.416560 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:45.416531    2576 generic.go:356] "Generic (PLEG): container finished" podID="d3798411-d5e9-4818-bd39-6ce8743d5b1e" containerID="5b6730c452caec396d49f1e8dd991ae62166fc40455e871b7aa1e1449faab2a9" exitCode=0
Aug 02 11:33:45.416746 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:45.416610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7333-139910175028688" event={"ID":"d3798411-d5e9-4818-bd39-6ce8743d5b1e","Type":"ContainerDied","Data":"5b6730c452caec396d49f1e8dd991ae62166fc40455e871b7aa1e1449faab2a9"}
Aug 02 11:33:46.592593 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:46.592569    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175028688"
Aug 02 11:33:46.610866 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:46.610830    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7333-139910175028688_d3798411-d5e9-4818-bd39-6ce8743d5b1e/test-curl-7333-139910175028688/0.log"
Aug 02 11:33:46.614101 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:46.614073    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhvg9\" (UniqueName: \"kubernetes.io/projected/d3798411-d5e9-4818-bd39-6ce8743d5b1e-kube-api-access-fhvg9\") pod \"d3798411-d5e9-4818-bd39-6ce8743d5b1e\" (UID: \"d3798411-d5e9-4818-bd39-6ce8743d5b1e\") "
Aug 02 11:33:46.616501 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:46.616469    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3798411-d5e9-4818-bd39-6ce8743d5b1e-kube-api-access-fhvg9" (OuterVolumeSpecName: "kube-api-access-fhvg9") pod "d3798411-d5e9-4818-bd39-6ce8743d5b1e" (UID: "d3798411-d5e9-4818-bd39-6ce8743d5b1e"). InnerVolumeSpecName "kube-api-access-fhvg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:33:46.636042 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:46.635996    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7333-139910175028688"]
Aug 02 11:33:46.640130 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:46.640106    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7333-139910175028688"]
Aug 02 11:33:46.715367 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:46.715279    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhvg9\" (UniqueName: \"kubernetes.io/projected/d3798411-d5e9-4818-bd39-6ce8743d5b1e-kube-api-access-fhvg9\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:33:46.951155 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:46.951120    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3798411-d5e9-4818-bd39-6ce8743d5b1e" path="/var/lib/kubelet/pods/d3798411-d5e9-4818-bd39-6ce8743d5b1e/volumes"
Aug 02 11:33:47.429586 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:47.429559    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7333-139910175028688"
Aug 02 11:33:47.429777 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:47.429603    2576 scope.go:117] "RemoveContainer" containerID="5b6730c452caec396d49f1e8dd991ae62166fc40455e871b7aa1e1449faab2a9"
Aug 02 11:33:54.286142 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.286105    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"]
Aug 02 11:33:54.286849 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.286829    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3798411-d5e9-4818-bd39-6ce8743d5b1e" containerName="test-curl-7333-139910175028688"
Aug 02 11:33:54.286849 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.286850    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3798411-d5e9-4818-bd39-6ce8743d5b1e" containerName="test-curl-7333-139910175028688"
Aug 02 11:33:54.287017 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.286961    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3798411-d5e9-4818-bd39-6ce8743d5b1e" containerName="test-curl-7333-139910175028688"
Aug 02 11:33:54.292974 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.292950    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.296814 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.296400    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-fe62c7-serving-cert\""
Aug 02 11:33:54.297788 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.297759    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"]
Aug 02 11:33:54.379295 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.379257    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6741eb5-0db7-46e0-b2e3-00aaac63aede-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.379504 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.379472    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwdnh\" (UniqueName: \"kubernetes.io/projected/f6741eb5-0db7-46e0-b2e3-00aaac63aede-kube-api-access-dwdnh\") pod \"maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.379609 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.379546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6741eb5-0db7-46e0-b2e3-00aaac63aede-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.480610 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.480573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwdnh\" (UniqueName: \"kubernetes.io/projected/f6741eb5-0db7-46e0-b2e3-00aaac63aede-kube-api-access-dwdnh\") pod \"maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.480777 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.480662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6741eb5-0db7-46e0-b2e3-00aaac63aede-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.480777 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.480714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6741eb5-0db7-46e0-b2e3-00aaac63aede-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.481470 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.481397    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6741eb5-0db7-46e0-b2e3-00aaac63aede-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.483301 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.483271    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6741eb5-0db7-46e0-b2e3-00aaac63aede-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.488508 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.488487    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwdnh\" (UniqueName: \"kubernetes.io/projected/f6741eb5-0db7-46e0-b2e3-00aaac63aede-kube-api-access-dwdnh\") pod \"maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.609580 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.609252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:54.763688 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:54.763655    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"]
Aug 02 11:33:54.765488 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:33:54.765461    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6741eb5_0db7_46e0_b2e3_00aaac63aede.slice/crio-5799dfd9139386c655fe8b65aa2a12ea202c3c3b5d0d35c4d1672139f789f930 WatchSource:0}: Error finding container 5799dfd9139386c655fe8b65aa2a12ea202c3c3b5d0d35c4d1672139f789f930: Status 404 returned error can't find the container with id 5799dfd9139386c655fe8b65aa2a12ea202c3c3b5d0d35c4d1672139f789f930
Aug 02 11:33:55.479369 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:55.479329    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s" event={"ID":"f6741eb5-0db7-46e0-b2e3-00aaac63aede","Type":"ContainerStarted","Data":"1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910"}
Aug 02 11:33:55.479746 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:55.479379    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s" event={"ID":"f6741eb5-0db7-46e0-b2e3-00aaac63aede","Type":"ContainerStarted","Data":"5799dfd9139386c655fe8b65aa2a12ea202c3c3b5d0d35c4d1672139f789f930"}
Aug 02 11:33:55.479746 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:55.479410    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:33:55.495505 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:33:55.495457    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s" podStartSLOduration=0.904979389 podStartE2EDuration="1.495435646s" totalImagesPullingTime="590.456257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:33:55.494398935 +0000 UTC m=+3637.024981565" watchObservedRunningTime="2026-08-02 11:33:55.495435646 +0000 UTC m=+3637.026018276"
Aug 02 11:34:01.490354 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:01.490323    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:34:19.099097 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.099055    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"]
Aug 02 11:34:19.103502 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.103482    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.106200 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.106176    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-9b1f20-openshift-default-dockercfg-jzhx2\""
Aug 02 11:34:19.110221 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.110194    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"]
Aug 02 11:34:19.210708 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.210660    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-data\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.210708 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.210711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-certs\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.210965 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.210752    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2gbn\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-kube-api-access-f2gbn\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.210965 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.210781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-token\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.210965 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.210879    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-envoy\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.210965 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.210929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-podinfo\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.211192 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.210988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a92d6d-e10d-4791-b04b-a5732dc701f9-istiod-ca-cert\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.211192 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.211061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-credential-socket\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.211192 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.211095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-socket\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.311715 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.311680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-credential-socket\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.311909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.311727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-socket\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.311909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.311780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-data\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.311909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.311806    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-certs\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.311909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.311844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2gbn\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-kube-api-access-f2gbn\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.311909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.311872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-token\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.312206 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.311922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-envoy\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.312206 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.311962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-podinfo\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.312206 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.312020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a92d6d-e10d-4791-b04b-a5732dc701f9-istiod-ca-cert\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.312206 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.312124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-credential-socket\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.312206 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.312172    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-socket\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.312525 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.312273    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-data\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.312525 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.312470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-certs\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.312753 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.312733    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a92d6d-e10d-4791-b04b-a5732dc701f9-istiod-ca-cert\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.314629 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.314606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-envoy\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.314796 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.314779    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-podinfo\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.328781 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.328756    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-token\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.328972 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.328951    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2gbn\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-kube-api-access-f2gbn\") pod \"e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") " pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.419180 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.419145    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:19.576965 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.576864    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"]
Aug 02 11:34:19.579278 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:34:19.579251    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a92d6d_e10d_4791_b04b_a5732dc701f9.slice/crio-9f7c3c919d6b9db8e10028b2f9197604b89135b38e1b26503eac5b17fa6b846f WatchSource:0}: Error finding container 9f7c3c919d6b9db8e10028b2f9197604b89135b38e1b26503eac5b17fa6b846f: Status 404 returned error can't find the container with id 9f7c3c919d6b9db8e10028b2f9197604b89135b38e1b26503eac5b17fa6b846f
Aug 02 11:34:19.581461 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.581433    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:34:19.581532 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.581495    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:34:19.581532 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.581523    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:34:19.628999 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:19.628975    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" event={"ID":"44a92d6d-e10d-4791-b04b-a5732dc701f9","Type":"ContainerStarted","Data":"9f7c3c919d6b9db8e10028b2f9197604b89135b38e1b26503eac5b17fa6b846f"}
Aug 02 11:34:20.636010 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:20.635973    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" event={"ID":"44a92d6d-e10d-4791-b04b-a5732dc701f9","Type":"ContainerStarted","Data":"00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171"}
Aug 02 11:34:20.658107 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:20.658059    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" podStartSLOduration=1.65804288 podStartE2EDuration="1.65804288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:34:20.655752673 +0000 UTC m=+3662.186335318" watchObservedRunningTime="2026-08-02 11:34:20.65804288 +0000 UTC m=+3662.188625505"
Aug 02 11:34:21.419815 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:21.419777    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:21.424754 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:21.424731    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:21.642889 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:21.642853    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:21.643739 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:21.643717    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:23.611574 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.611542    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"]
Aug 02 11:34:23.615574 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.615550    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:23.618284 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.618262    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-9b1f20-serving-cert\""
Aug 02 11:34:23.626752 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.625881    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"]
Aug 02 11:34:23.754485 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.754448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:23.754692 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.754518    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgj9k\" (UniqueName: \"kubernetes.io/projected/7e1f4043-7053-4c54-83f3-b91a7c4befa2-kube-api-access-sgj9k\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:23.754692 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.754625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e1f4043-7053-4c54-83f3-b91a7c4befa2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:23.855187 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.855152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:23.855391 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.855207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgj9k\" (UniqueName: \"kubernetes.io/projected/7e1f4043-7053-4c54-83f3-b91a7c4befa2-kube-api-access-sgj9k\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:23.855391 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.855292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e1f4043-7053-4c54-83f3-b91a7c4befa2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:23.855391 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:34:23.855304    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-serving-cert: secret "maas-api-e2e-ipp-b-9b1f20-serving-cert" not found
Aug 02 11:34:23.855391 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:34:23.855383    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls podName:7e1f4043-7053-4c54-83f3-b91a7c4befa2 nodeName:}" failed. No retries permitted until 2026-08-02 11:34:24.35536348 +0000 UTC m=+3665.885946088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls") pod "maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp" (UID: "7e1f4043-7053-4c54-83f3-b91a7c4befa2") : secret "maas-api-e2e-ipp-b-9b1f20-serving-cert" not found
Aug 02 11:34:23.855926 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.855909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e1f4043-7053-4c54-83f3-b91a7c4befa2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:23.863882 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:23.863829    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgj9k\" (UniqueName: \"kubernetes.io/projected/7e1f4043-7053-4c54-83f3-b91a7c4befa2-kube-api-access-sgj9k\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:24.359484 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:24.359442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:24.359760 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:34:24.359590    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-serving-cert: secret "maas-api-e2e-ipp-b-9b1f20-serving-cert" not found
Aug 02 11:34:24.359760 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:34:24.359655    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls podName:7e1f4043-7053-4c54-83f3-b91a7c4befa2 nodeName:}" failed. No retries permitted until 2026-08-02 11:34:25.35964007 +0000 UTC m=+3666.890222677 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls") pod "maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp" (UID: "7e1f4043-7053-4c54-83f3-b91a7c4befa2") : secret "maas-api-e2e-ipp-b-9b1f20-serving-cert" not found
Aug 02 11:34:25.371815 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:25.371778    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:25.374337 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:25.374310    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:25.432092 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:25.432059    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:25.576200 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:25.576172    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"]
Aug 02 11:34:25.577142 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:34:25.577121    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1f4043_7053_4c54_83f3_b91a7c4befa2.slice/crio-2c0109fc100bd2e80645afd2b60a7bb342389215d3240644acce41478fcfdd62 WatchSource:0}: Error finding container 2c0109fc100bd2e80645afd2b60a7bb342389215d3240644acce41478fcfdd62: Status 404 returned error can't find the container with id 2c0109fc100bd2e80645afd2b60a7bb342389215d3240644acce41478fcfdd62
Aug 02 11:34:25.668253 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:25.668218    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp" event={"ID":"7e1f4043-7053-4c54-83f3-b91a7c4befa2","Type":"ContainerStarted","Data":"2c0109fc100bd2e80645afd2b60a7bb342389215d3240644acce41478fcfdd62"}
Aug 02 11:34:26.675792 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:26.675748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp" event={"ID":"7e1f4043-7053-4c54-83f3-b91a7c4befa2","Type":"ContainerStarted","Data":"04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8"}
Aug 02 11:34:26.676276 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:26.675966    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:26.694411 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:26.694370    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp" podStartSLOduration=3.299780566 podStartE2EDuration="3.694357716s" totalImagesPullingTime="394.57715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:34:26.691053949 +0000 UTC m=+3668.221636601" watchObservedRunningTime="2026-08-02 11:34:26.694357716 +0000 UTC m=+3668.224940344"
Aug 02 11:34:32.692339 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:32.692305    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:34:37.173218 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:37.172397    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": read tcp 10.133.0.2:55758->10.133.0.84:15021: read: connection reset by peer" start-of-body=
Aug 02 11:34:37.173218 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:37.172474    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": read tcp 10.133.0.2:55758->10.133.0.84:15021: read: connection reset by peer"
Aug 02 11:34:37.761959 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:37.761929    2576 generic.go:356] "Generic (PLEG): container finished" podID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerID="00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171" exitCode=0
Aug 02 11:34:37.762157 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:37.761992    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" event={"ID":"44a92d6d-e10d-4791-b04b-a5732dc701f9","Type":"ContainerDied","Data":"00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171"}
Aug 02 11:34:37.762398 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:37.762384    2576 scope.go:117] "RemoveContainer" containerID="00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171"
Aug 02 11:34:37.763454 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:37.763425    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:34:37.763516 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:37.763502    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:34:37.763560 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:37.763538    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Aug 02 11:34:38.770322 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:38.770288    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" event={"ID":"44a92d6d-e10d-4791-b04b-a5732dc701f9","Type":"ContainerStarted","Data":"9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5"}
Aug 02 11:34:39.419727 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:39.419690    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:39.419936 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:39.419913    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Aug 02 11:34:39.419981 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:39.419964    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Aug 02 11:34:40.420374 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:40.420334    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Aug 02 11:34:40.420850 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:40.420399    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Aug 02 11:34:41.419968 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:41.419932    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Aug 02 11:34:41.420179 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:41.420001    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Aug 02 11:34:42.419924 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:42.419887    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Aug 02 11:34:42.420325 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:42.419947    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Aug 02 11:34:43.419662 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:43.419624    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Aug 02 11:34:43.419854 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:43.419681    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Aug 02 11:34:44.422909 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:44.422872    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:44.423320 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:44.423242    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:44.423975 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:44.423958    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:34:54.290908 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:34:54.290870    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w6bc7_1787e6d1-3ebf-4824-9540-88d0b2022fd6/payload-processing/0.log"
Aug 02 11:35:39.136553 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:39.136521    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w6bc7_1787e6d1-3ebf-4824-9540-88d0b2022fd6/payload-processing/0.log"
Aug 02 11:35:43.792502 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:43.792468    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"]
Aug 02 11:35:43.796476 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:43.796456    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:43.799275 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:43.798992    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-52c040-serving-cert\""
Aug 02 11:35:43.804353 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:43.804328    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"]
Aug 02 11:35:43.921358 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:43.921322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxwrc\" (UniqueName: \"kubernetes.io/projected/d553ebcb-9210-4414-b3a3-407442344309-kube-api-access-zxwrc\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:43.921571 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:43.921398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d553ebcb-9210-4414-b3a3-407442344309-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:43.921571 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:43.921471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d553ebcb-9210-4414-b3a3-407442344309-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:44.022560 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:44.022519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxwrc\" (UniqueName: \"kubernetes.io/projected/d553ebcb-9210-4414-b3a3-407442344309-kube-api-access-zxwrc\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:44.022765 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:44.022604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d553ebcb-9210-4414-b3a3-407442344309-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:44.022765 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:44.022656    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d553ebcb-9210-4414-b3a3-407442344309-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:44.022765 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:35:44.022740    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-serving-cert: secret "maas-api-e2e-ipp-cleanup-52c040-serving-cert" not found
Aug 02 11:35:44.022943 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:35:44.022810    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d553ebcb-9210-4414-b3a3-407442344309-maas-api-tls podName:d553ebcb-9210-4414-b3a3-407442344309 nodeName:}" failed. No retries permitted until 2026-08-02 11:35:44.522789494 +0000 UTC m=+3746.053372101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d553ebcb-9210-4414-b3a3-407442344309-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254" (UID: "d553ebcb-9210-4414-b3a3-407442344309") : secret "maas-api-e2e-ipp-cleanup-52c040-serving-cert" not found
Aug 02 11:35:44.023395 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:44.023375    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d553ebcb-9210-4414-b3a3-407442344309-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:44.031350 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:44.031331    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxwrc\" (UniqueName: \"kubernetes.io/projected/d553ebcb-9210-4414-b3a3-407442344309-kube-api-access-zxwrc\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:44.527067 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:44.527014    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d553ebcb-9210-4414-b3a3-407442344309-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:44.529961 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:44.529936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d553ebcb-9210-4414-b3a3-407442344309-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:44.713279 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:44.713243    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:44.857073 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:44.857040    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"]
Aug 02 11:35:44.858391 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:35:44.858364    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd553ebcb_9210_4414_b3a3_407442344309.slice/crio-e10aba52f5a46ec77e3472aa5695f3437d25626509e4d1bb0726a26acd747f73 WatchSource:0}: Error finding container e10aba52f5a46ec77e3472aa5695f3437d25626509e4d1bb0726a26acd747f73: Status 404 returned error can't find the container with id e10aba52f5a46ec77e3472aa5695f3437d25626509e4d1bb0726a26acd747f73
Aug 02 11:35:45.173183 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:45.173146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254" event={"ID":"d553ebcb-9210-4414-b3a3-407442344309","Type":"ContainerStarted","Data":"e10aba52f5a46ec77e3472aa5695f3437d25626509e4d1bb0726a26acd747f73"}
Aug 02 11:35:46.181492 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:46.181456    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254" event={"ID":"d553ebcb-9210-4414-b3a3-407442344309","Type":"ContainerStarted","Data":"490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409"}
Aug 02 11:35:46.181886 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:46.181531    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:35:46.199017 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:46.198973    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254" podStartSLOduration=2.827911844 podStartE2EDuration="3.198955532s" totalImagesPullingTime="371.043688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-02 11:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:35:46.196672856 +0000 UTC m=+3747.727255489" watchObservedRunningTime="2026-08-02 11:35:46.198955532 +0000 UTC m=+3747.729538160"
Aug 02 11:35:52.192898 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:35:52.192868    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:36:08.744644 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:08.744289    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"]
Aug 02 11:36:08.744644 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:08.744536    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254" podUID="d553ebcb-9210-4414-b3a3-407442344309" containerName="maas-api" containerID="cri-o://490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409" gracePeriod=30
Aug 02 11:36:09.009704 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.009644    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:36:09.056514 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.056481    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxwrc\" (UniqueName: \"kubernetes.io/projected/d553ebcb-9210-4414-b3a3-407442344309-kube-api-access-zxwrc\") pod \"d553ebcb-9210-4414-b3a3-407442344309\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") "
Aug 02 11:36:09.056679 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.056524    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d553ebcb-9210-4414-b3a3-407442344309-openshift-service-ca\") pod \"d553ebcb-9210-4414-b3a3-407442344309\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") "
Aug 02 11:36:09.056679 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.056587    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d553ebcb-9210-4414-b3a3-407442344309-maas-api-tls\") pod \"d553ebcb-9210-4414-b3a3-407442344309\" (UID: \"d553ebcb-9210-4414-b3a3-407442344309\") "
Aug 02 11:36:09.056914 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.056884    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d553ebcb-9210-4414-b3a3-407442344309-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d553ebcb-9210-4414-b3a3-407442344309" (UID: "d553ebcb-9210-4414-b3a3-407442344309"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:36:09.057147 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.057112    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d553ebcb-9210-4414-b3a3-407442344309-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:36:09.058908 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.058884    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d553ebcb-9210-4414-b3a3-407442344309-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d553ebcb-9210-4414-b3a3-407442344309" (UID: "d553ebcb-9210-4414-b3a3-407442344309"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:36:09.058979 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.058934    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d553ebcb-9210-4414-b3a3-407442344309-kube-api-access-zxwrc" (OuterVolumeSpecName: "kube-api-access-zxwrc") pod "d553ebcb-9210-4414-b3a3-407442344309" (UID: "d553ebcb-9210-4414-b3a3-407442344309"). InnerVolumeSpecName "kube-api-access-zxwrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:36:09.157649 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.157620    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d553ebcb-9210-4414-b3a3-407442344309-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:36:09.157649 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.157645    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxwrc\" (UniqueName: \"kubernetes.io/projected/d553ebcb-9210-4414-b3a3-407442344309-kube-api-access-zxwrc\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:36:09.319915 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.319822    2576 generic.go:356] "Generic (PLEG): container finished" podID="d553ebcb-9210-4414-b3a3-407442344309" containerID="490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409" exitCode=0
Aug 02 11:36:09.319915 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.319866    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"
Aug 02 11:36:09.320199 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.319914    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254" event={"ID":"d553ebcb-9210-4414-b3a3-407442344309","Type":"ContainerDied","Data":"490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409"}
Aug 02 11:36:09.320199 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.319975    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254" event={"ID":"d553ebcb-9210-4414-b3a3-407442344309","Type":"ContainerDied","Data":"e10aba52f5a46ec77e3472aa5695f3437d25626509e4d1bb0726a26acd747f73"}
Aug 02 11:36:09.320199 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.320004    2576 scope.go:117] "RemoveContainer" containerID="490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409"
Aug 02 11:36:09.330188 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.330171    2576 scope.go:117] "RemoveContainer" containerID="490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409"
Aug 02 11:36:09.330451 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:36:09.330429    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409\": container with ID starting with 490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409 not found: ID does not exist" containerID="490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409"
Aug 02 11:36:09.330509 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.330484    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409"} err="failed to get container status \"490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409\": rpc error: code = NotFound desc = could not find container \"490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409\": container with ID starting with 490242a698119efd0d65c7a25c27f39e232d98f1524bcb1a3adc87b4429d9409 not found: ID does not exist"
Aug 02 11:36:09.341904 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.341884    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"]
Aug 02 11:36:09.347240 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:09.347222    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-52c040-5c6d6bfdb8-gh254"]
Aug 02 11:36:10.950721 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:10.950687    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d553ebcb-9210-4414-b3a3-407442344309" path="/var/lib/kubelet/pods/d553ebcb-9210-4414-b3a3-407442344309/volumes"
Aug 02 11:36:24.114950 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.114911    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"]
Aug 02 11:36:24.115438 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.115169    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s" podUID="f6741eb5-0db7-46e0-b2e3-00aaac63aede" containerName="maas-api" containerID="cri-o://1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910" gracePeriod=30
Aug 02 11:36:24.388775 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.388745    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:36:24.412918 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.412872    2576 generic.go:356] "Generic (PLEG): container finished" podID="f6741eb5-0db7-46e0-b2e3-00aaac63aede" containerID="1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910" exitCode=0
Aug 02 11:36:24.413126 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.412949    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"
Aug 02 11:36:24.413126 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.412962    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s" event={"ID":"f6741eb5-0db7-46e0-b2e3-00aaac63aede","Type":"ContainerDied","Data":"1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910"}
Aug 02 11:36:24.413126 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.413011    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s" event={"ID":"f6741eb5-0db7-46e0-b2e3-00aaac63aede","Type":"ContainerDied","Data":"5799dfd9139386c655fe8b65aa2a12ea202c3c3b5d0d35c4d1672139f789f930"}
Aug 02 11:36:24.413126 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.413083    2576 scope.go:117] "RemoveContainer" containerID="1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910"
Aug 02 11:36:24.428251 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.428231    2576 scope.go:117] "RemoveContainer" containerID="1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910"
Aug 02 11:36:24.428510 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:36:24.428492    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910\": container with ID starting with 1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910 not found: ID does not exist" containerID="1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910"
Aug 02 11:36:24.428569 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.428522    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910"} err="failed to get container status \"1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910\": rpc error: code = NotFound desc = could not find container \"1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910\": container with ID starting with 1d4484cc5a59dae31c8bb7869338a94f736a5359575f0c1084143012ea489910 not found: ID does not exist"
Aug 02 11:36:24.503877 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.503845    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwdnh\" (UniqueName: \"kubernetes.io/projected/f6741eb5-0db7-46e0-b2e3-00aaac63aede-kube-api-access-dwdnh\") pod \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") "
Aug 02 11:36:24.503877 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.503878    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6741eb5-0db7-46e0-b2e3-00aaac63aede-openshift-service-ca\") pod \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") "
Aug 02 11:36:24.504137 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.503921    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6741eb5-0db7-46e0-b2e3-00aaac63aede-maas-api-tls\") pod \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\" (UID: \"f6741eb5-0db7-46e0-b2e3-00aaac63aede\") "
Aug 02 11:36:24.504326 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.504300    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6741eb5-0db7-46e0-b2e3-00aaac63aede-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f6741eb5-0db7-46e0-b2e3-00aaac63aede" (UID: "f6741eb5-0db7-46e0-b2e3-00aaac63aede"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:36:24.506156 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.506123    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6741eb5-0db7-46e0-b2e3-00aaac63aede-kube-api-access-dwdnh" (OuterVolumeSpecName: "kube-api-access-dwdnh") pod "f6741eb5-0db7-46e0-b2e3-00aaac63aede" (UID: "f6741eb5-0db7-46e0-b2e3-00aaac63aede"). InnerVolumeSpecName "kube-api-access-dwdnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:36:24.506261 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.506199    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6741eb5-0db7-46e0-b2e3-00aaac63aede-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f6741eb5-0db7-46e0-b2e3-00aaac63aede" (UID: "f6741eb5-0db7-46e0-b2e3-00aaac63aede"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:36:24.605202 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.605168    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwdnh\" (UniqueName: \"kubernetes.io/projected/f6741eb5-0db7-46e0-b2e3-00aaac63aede-kube-api-access-dwdnh\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:36:24.605202 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.605199    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6741eb5-0db7-46e0-b2e3-00aaac63aede-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:36:24.605202 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.605210    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6741eb5-0db7-46e0-b2e3-00aaac63aede-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:36:24.736194 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.736160    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"]
Aug 02 11:36:24.741888 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.741863    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fe62c7-77d79f9ccd-wqh6s"]
Aug 02 11:36:24.950566 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:36:24.950533    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6741eb5-0db7-46e0-b2e3-00aaac63aede" path="/var/lib/kubelet/pods/f6741eb5-0db7-46e0-b2e3-00aaac63aede/volumes"
Aug 02 11:37:14.120821 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.120782    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"]
Aug 02 11:37:14.121365 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.121095    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp" podUID="7e1f4043-7053-4c54-83f3-b91a7c4befa2" containerName="maas-api" containerID="cri-o://04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8" gracePeriod=30
Aug 02 11:37:14.389517 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.389486    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:37:14.464990 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.464950    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls\") pod \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") "
Aug 02 11:37:14.464990 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.464995    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgj9k\" (UniqueName: \"kubernetes.io/projected/7e1f4043-7053-4c54-83f3-b91a7c4befa2-kube-api-access-sgj9k\") pod \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") "
Aug 02 11:37:14.465262 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.465018    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e1f4043-7053-4c54-83f3-b91a7c4befa2-openshift-service-ca\") pod \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\" (UID: \"7e1f4043-7053-4c54-83f3-b91a7c4befa2\") "
Aug 02 11:37:14.465646 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.465609    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e1f4043-7053-4c54-83f3-b91a7c4befa2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e1f4043-7053-4c54-83f3-b91a7c4befa2" (UID: "7e1f4043-7053-4c54-83f3-b91a7c4befa2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:37:14.467379 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.467341    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e1f4043-7053-4c54-83f3-b91a7c4befa2" (UID: "7e1f4043-7053-4c54-83f3-b91a7c4befa2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:37:14.467516 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.467490    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e1f4043-7053-4c54-83f3-b91a7c4befa2-kube-api-access-sgj9k" (OuterVolumeSpecName: "kube-api-access-sgj9k") pod "7e1f4043-7053-4c54-83f3-b91a7c4befa2" (UID: "7e1f4043-7053-4c54-83f3-b91a7c4befa2"). InnerVolumeSpecName "kube-api-access-sgj9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:37:14.565980 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.565945    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e1f4043-7053-4c54-83f3-b91a7c4befa2-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:14.565980 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.565976    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgj9k\" (UniqueName: \"kubernetes.io/projected/7e1f4043-7053-4c54-83f3-b91a7c4befa2-kube-api-access-sgj9k\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:14.565980 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.565988    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e1f4043-7053-4c54-83f3-b91a7c4befa2-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:14.708362 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.708328    2576 generic.go:356] "Generic (PLEG): container finished" podID="7e1f4043-7053-4c54-83f3-b91a7c4befa2" containerID="04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8" exitCode=0
Aug 02 11:37:14.708536 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.708424    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"
Aug 02 11:37:14.708536 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.708464    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp" event={"ID":"7e1f4043-7053-4c54-83f3-b91a7c4befa2","Type":"ContainerDied","Data":"04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8"}
Aug 02 11:37:14.708536 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.708514    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp" event={"ID":"7e1f4043-7053-4c54-83f3-b91a7c4befa2","Type":"ContainerDied","Data":"2c0109fc100bd2e80645afd2b60a7bb342389215d3240644acce41478fcfdd62"}
Aug 02 11:37:14.708536 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.708538    2576 scope.go:117] "RemoveContainer" containerID="04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8"
Aug 02 11:37:14.718753 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.718736    2576 scope.go:117] "RemoveContainer" containerID="04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8"
Aug 02 11:37:14.718993 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:37:14.718974    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8\": container with ID starting with 04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8 not found: ID does not exist" containerID="04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8"
Aug 02 11:37:14.719078 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.719006    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8"} err="failed to get container status \"04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8\": rpc error: code = NotFound desc = could not find container \"04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8\": container with ID starting with 04caa164f3ba2f42788d066c12021c7c6e66e7c51b56eea347eddb0cc07ad7b8 not found: ID does not exist"
Aug 02 11:37:14.730053 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.730008    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"]
Aug 02 11:37:14.733551 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.733530    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9b1f20-7d8fddbc44-nhcmp"]
Aug 02 11:37:14.952655 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:14.952619    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e1f4043-7053-4c54-83f3-b91a7c4befa2" path="/var/lib/kubelet/pods/7e1f4043-7053-4c54-83f3-b91a7c4befa2/volumes"
Aug 02 11:37:20.836478 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:20.836447    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"]
Aug 02 11:37:20.836930 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:20.836648    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy" containerID="cri-o://9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5" gracePeriod=30
Aug 02 11:37:21.756323 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:21.756289    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"]
Aug 02 11:37:21.756914 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:21.756853    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx" podUID="f2d1e3b6-d262-46a6-b834-e6288230e9b4" containerName="maas-api" containerID="cri-o://c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c" gracePeriod=30
Aug 02 11:37:22.034077 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.034052    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:37:22.148958 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.148926    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph8hb\" (UniqueName: \"kubernetes.io/projected/f2d1e3b6-d262-46a6-b834-e6288230e9b4-kube-api-access-ph8hb\") pod \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") "
Aug 02 11:37:22.149167 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.148971    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d1e3b6-d262-46a6-b834-e6288230e9b4-maas-api-tls\") pod \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") "
Aug 02 11:37:22.149167 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.148998    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d1e3b6-d262-46a6-b834-e6288230e9b4-openshift-service-ca\") pod \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\" (UID: \"f2d1e3b6-d262-46a6-b834-e6288230e9b4\") "
Aug 02 11:37:22.149375 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.149350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2d1e3b6-d262-46a6-b834-e6288230e9b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f2d1e3b6-d262-46a6-b834-e6288230e9b4" (UID: "f2d1e3b6-d262-46a6-b834-e6288230e9b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:37:22.151189 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.151161    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2d1e3b6-d262-46a6-b834-e6288230e9b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f2d1e3b6-d262-46a6-b834-e6288230e9b4" (UID: "f2d1e3b6-d262-46a6-b834-e6288230e9b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:37:22.151304 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.151206    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d1e3b6-d262-46a6-b834-e6288230e9b4-kube-api-access-ph8hb" (OuterVolumeSpecName: "kube-api-access-ph8hb") pod "f2d1e3b6-d262-46a6-b834-e6288230e9b4" (UID: "f2d1e3b6-d262-46a6-b834-e6288230e9b4"). InnerVolumeSpecName "kube-api-access-ph8hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:37:22.250207 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.250171    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d1e3b6-d262-46a6-b834-e6288230e9b4-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:22.250207 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.250200    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d1e3b6-d262-46a6-b834-e6288230e9b4-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:22.250408 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.250231    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph8hb\" (UniqueName: \"kubernetes.io/projected/f2d1e3b6-d262-46a6-b834-e6288230e9b4-kube-api-access-ph8hb\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:22.765997 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.765960    2576 generic.go:356] "Generic (PLEG): container finished" podID="f2d1e3b6-d262-46a6-b834-e6288230e9b4" containerID="c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c" exitCode=0
Aug 02 11:37:22.766217 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.766003    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"
Aug 02 11:37:22.766217 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.766053    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx" event={"ID":"f2d1e3b6-d262-46a6-b834-e6288230e9b4","Type":"ContainerDied","Data":"c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c"}
Aug 02 11:37:22.766217 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.766112    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx" event={"ID":"f2d1e3b6-d262-46a6-b834-e6288230e9b4","Type":"ContainerDied","Data":"1bf95abef1d6b7d3602746fec6a6ead015982d75d0d0bd3fdbb830764ebe4461"}
Aug 02 11:37:22.766217 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.766138    2576 scope.go:117] "RemoveContainer" containerID="c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c"
Aug 02 11:37:22.775790 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.775773    2576 scope.go:117] "RemoveContainer" containerID="c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c"
Aug 02 11:37:22.776076 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:37:22.776055    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c\": container with ID starting with c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c not found: ID does not exist" containerID="c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c"
Aug 02 11:37:22.776133 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.776085    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c"} err="failed to get container status \"c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c\": rpc error: code = NotFound desc = could not find container \"c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c\": container with ID starting with c7e61f352a3f02a3332d2e4ccb91a8f1d40c1b3326516489b8fc5dda00cd538c not found: ID does not exist"
Aug 02 11:37:22.788127 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.788100    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"]
Aug 02 11:37:22.791203 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.791182    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3951f6-664dbb677f-wpbzx"]
Aug 02 11:37:22.957886 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:22.957846    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2d1e3b6-d262-46a6-b834-e6288230e9b4" path="/var/lib/kubelet/pods/f2d1e3b6-d262-46a6-b834-e6288230e9b4/volumes"
Aug 02 11:37:26.096301 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.096272    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:37:26.187797 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.187763    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2gbn\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-kube-api-access-f2gbn\") pod \"44a92d6d-e10d-4791-b04b-a5732dc701f9\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") "
Aug 02 11:37:26.187974 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.187823    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a92d6d-e10d-4791-b04b-a5732dc701f9-istiod-ca-cert\") pod \"44a92d6d-e10d-4791-b04b-a5732dc701f9\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") "
Aug 02 11:37:26.187974 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.187861    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-data\") pod \"44a92d6d-e10d-4791-b04b-a5732dc701f9\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") "
Aug 02 11:37:26.187974 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.187886    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-socket\") pod \"44a92d6d-e10d-4791-b04b-a5732dc701f9\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") "
Aug 02 11:37:26.188283 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.188258    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44a92d6d-e10d-4791-b04b-a5732dc701f9" (UID: "44a92d6d-e10d-4791-b04b-a5732dc701f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:26.188400 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.188261    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-envoy\") pod \"44a92d6d-e10d-4791-b04b-a5732dc701f9\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") "
Aug 02 11:37:26.188400 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.188322    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "44a92d6d-e10d-4791-b04b-a5732dc701f9" (UID: "44a92d6d-e10d-4791-b04b-a5732dc701f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:26.188400 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.188332    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44a92d6d-e10d-4791-b04b-a5732dc701f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44a92d6d-e10d-4791-b04b-a5732dc701f9" (UID: "44a92d6d-e10d-4791-b04b-a5732dc701f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:37:26.188400 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.188339    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-podinfo\") pod \"44a92d6d-e10d-4791-b04b-a5732dc701f9\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") "
Aug 02 11:37:26.188616 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.188402    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-credential-socket\") pod \"44a92d6d-e10d-4791-b04b-a5732dc701f9\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") "
Aug 02 11:37:26.188732 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.188453    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-token\") pod \"44a92d6d-e10d-4791-b04b-a5732dc701f9\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") "
Aug 02 11:37:26.189071 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.188732    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44a92d6d-e10d-4791-b04b-a5732dc701f9" (UID: "44a92d6d-e10d-4791-b04b-a5732dc701f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:26.189071 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.188875    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-certs\") pod \"44a92d6d-e10d-4791-b04b-a5732dc701f9\" (UID: \"44a92d6d-e10d-4791-b04b-a5732dc701f9\") "
Aug 02 11:37:26.189302 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.189134    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44a92d6d-e10d-4791-b04b-a5732dc701f9" (UID: "44a92d6d-e10d-4791-b04b-a5732dc701f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:26.189524 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.189472    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-data\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:26.189623 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.189536    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:26.189623 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.189551    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-credential-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:26.189623 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.189564    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-workload-certs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:26.189623 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.189578    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a92d6d-e10d-4791-b04b-a5732dc701f9-istiod-ca-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:26.190732 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.190709    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44a92d6d-e10d-4791-b04b-a5732dc701f9" (UID: "44a92d6d-e10d-4791-b04b-a5732dc701f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:26.190948 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.190913    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-kube-api-access-f2gbn" (OuterVolumeSpecName: "kube-api-access-f2gbn") pod "44a92d6d-e10d-4791-b04b-a5732dc701f9" (UID: "44a92d6d-e10d-4791-b04b-a5732dc701f9"). InnerVolumeSpecName "kube-api-access-f2gbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:37:26.191083 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.190954    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "44a92d6d-e10d-4791-b04b-a5732dc701f9" (UID: "44a92d6d-e10d-4791-b04b-a5732dc701f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:37:26.191305 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.191286    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44a92d6d-e10d-4791-b04b-a5732dc701f9" (UID: "44a92d6d-e10d-4791-b04b-a5732dc701f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 02 11:37:26.290219 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.290184    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2gbn\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-kube-api-access-f2gbn\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:26.290219 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.290212    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-envoy\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:26.290219 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.290222    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-podinfo\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:26.290219 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.290230    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a92d6d-e10d-4791-b04b-a5732dc701f9-istio-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:26.791422 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.791388    2576 generic.go:356] "Generic (PLEG): container finished" podID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerID="9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5" exitCode=0
Aug 02 11:37:26.791623 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.791445    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"
Aug 02 11:37:26.791623 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.791475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" event={"ID":"44a92d6d-e10d-4791-b04b-a5732dc701f9","Type":"ContainerDied","Data":"9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5"}
Aug 02 11:37:26.791623 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.791533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5" event={"ID":"44a92d6d-e10d-4791-b04b-a5732dc701f9","Type":"ContainerDied","Data":"9f7c3c919d6b9db8e10028b2f9197604b89135b38e1b26503eac5b17fa6b846f"}
Aug 02 11:37:26.791623 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.791560    2576 scope.go:117] "RemoveContainer" containerID="9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5"
Aug 02 11:37:26.805999 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.805982    2576 scope.go:117] "RemoveContainer" containerID="00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171"
Aug 02 11:37:26.814842 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.814825    2576 scope.go:117] "RemoveContainer" containerID="9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5"
Aug 02 11:37:26.815181 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:37:26.815143    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5\": container with ID starting with 9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5 not found: ID does not exist" containerID="9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5"
Aug 02 11:37:26.815181 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.815172    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5"} err="failed to get container status \"9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5\": rpc error: code = NotFound desc = could not find container \"9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5\": container with ID starting with 9efddb70c942a33d66f95e553758293ed4c578838bdaba7fa5bcc3cc9c4ba9b5 not found: ID does not exist"
Aug 02 11:37:26.815343 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.815190    2576 scope.go:117] "RemoveContainer" containerID="00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171"
Aug 02 11:37:26.815569 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:37:26.815544    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171\": container with ID starting with 00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171 not found: ID does not exist" containerID="00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171"
Aug 02 11:37:26.815669 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.815583    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171"} err="failed to get container status \"00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171\": rpc error: code = NotFound desc = could not find container \"00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171\": container with ID starting with 00fd514fb9c54f11b2d5c7ab4d454ebe98bc847bde587440f010acdf89e78171 not found: ID does not exist"
Aug 02 11:37:26.815802 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.815778    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"]
Aug 02 11:37:26.818657 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.818640    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b1f20-openshift-default-555dcb68c6-67gz5"]
Aug 02 11:37:26.951245 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:26.951215    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" path="/var/lib/kubelet/pods/44a92d6d-e10d-4791-b04b-a5732dc701f9/volumes"
Aug 02 11:37:28.657782 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:28.657746    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"]
Aug 02 11:37:28.658289 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:28.658041    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy" containerID="cri-o://f13ce6b7a81d3f6da7672528eac8efaa9ab9f9e5268dacca7c920ae698eafa85" gracePeriod=30
Aug 02 11:37:29.695590 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:29.695555    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"]
Aug 02 11:37:29.696072 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:29.695806    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr" podUID="d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659" containerName="maas-api" containerID="cri-o://c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de" gracePeriod=30
Aug 02 11:37:29.961072 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:29.960982    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:37:30.024696 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.024665    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-openshift-service-ca\") pod \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") "
Aug 02 11:37:30.024883 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.024736    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b84kx\" (UniqueName: \"kubernetes.io/projected/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-kube-api-access-b84kx\") pod \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") "
Aug 02 11:37:30.024883 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.024777    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-maas-api-tls\") pod \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\" (UID: \"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659\") "
Aug 02 11:37:30.025117 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.025085    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659" (UID: "d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:37:30.027118 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.027095    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659" (UID: "d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 02 11:37:30.027118 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.027097    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-kube-api-access-b84kx" (OuterVolumeSpecName: "kube-api-access-b84kx") pod "d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659" (UID: "d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659"). InnerVolumeSpecName "kube-api-access-b84kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:37:30.125836 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.125800    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-openshift-service-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:30.125836 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.125829    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b84kx\" (UniqueName: \"kubernetes.io/projected/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-kube-api-access-b84kx\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:30.125836 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.125840    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659-maas-api-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:30.818513 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.818474    2576 generic.go:356] "Generic (PLEG): container finished" podID="d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659" containerID="c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de" exitCode=0
Aug 02 11:37:30.818893 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.818542    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"
Aug 02 11:37:30.818893 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.818558    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr" event={"ID":"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659","Type":"ContainerDied","Data":"c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de"}
Aug 02 11:37:30.818893 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.818641    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr" event={"ID":"d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659","Type":"ContainerDied","Data":"4dc69b0b1bc855c9c950b5992cc1b167aaacc7cc6d2d6dc51665922007cad0cb"}
Aug 02 11:37:30.818893 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.818665    2576 scope.go:117] "RemoveContainer" containerID="c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de"
Aug 02 11:37:30.829840 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.829819    2576 scope.go:117] "RemoveContainer" containerID="c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de"
Aug 02 11:37:30.830155 ip-10-0-135-193 kubenswrapper[2576]: E0802 11:37:30.830130    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de\": container with ID starting with c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de not found: ID does not exist" containerID="c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de"
Aug 02 11:37:30.830244 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.830162    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de"} err="failed to get container status \"c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de\": rpc error: code = NotFound desc = could not find container \"c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de\": container with ID starting with c39d6ed5e321234561eb271d0202ef532527d9bc2eaa398f4c6be0c2585653de not found: ID does not exist"
Aug 02 11:37:30.841263 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.841141    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"]
Aug 02 11:37:30.843914 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.843891    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4f1bb7-7c85f94974-n5qpr"]
Aug 02 11:37:30.951492 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:30.951461    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659" path="/var/lib/kubelet/pods/d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659/volumes"
Aug 02 11:37:33.842603 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.842572    2576 generic.go:356] "Generic (PLEG): container finished" podID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerID="f13ce6b7a81d3f6da7672528eac8efaa9ab9f9e5268dacca7c920ae698eafa85" exitCode=0
Aug 02 11:37:33.842984 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.842652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" event={"ID":"8e75a9e3-f08c-48a1-90e6-621cdcc99655","Type":"ContainerDied","Data":"f13ce6b7a81d3f6da7672528eac8efaa9ab9f9e5268dacca7c920ae698eafa85"}
Aug 02 11:37:33.842984 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.842732    2576 scope.go:117] "RemoveContainer" containerID="12db0ccd990ae2ad2e24696835b036714a0c47e6e3c6b5f813a706cf1cf66964"
Aug 02 11:37:33.917181 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.917158    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:37:33.962988 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.962899    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-socket\") pod \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") "
Aug 02 11:37:33.962988 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.962942    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-certs\") pod \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") "
Aug 02 11:37:33.962988 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.962979    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-credential-socket\") pod \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") "
Aug 02 11:37:33.963293 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963009    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-data\") pod \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") "
Aug 02 11:37:33.963293 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963052    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-envoy\") pod \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") "
Aug 02 11:37:33.963293 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963118    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-token\") pod \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") "
Aug 02 11:37:33.963293 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963176    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42jmw\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-kube-api-access-42jmw\") pod \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") "
Aug 02 11:37:33.963293 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963209    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-podinfo\") pod \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") "
Aug 02 11:37:33.963293 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963240    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istiod-ca-cert\") pod \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\" (UID: \"8e75a9e3-f08c-48a1-90e6-621cdcc99655\") "
Aug 02 11:37:33.963293 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963269    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8e75a9e3-f08c-48a1-90e6-621cdcc99655" (UID: "8e75a9e3-f08c-48a1-90e6-621cdcc99655"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:33.963642 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963367    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8e75a9e3-f08c-48a1-90e6-621cdcc99655" (UID: "8e75a9e3-f08c-48a1-90e6-621cdcc99655"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:33.963642 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963383    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-data" (OuterVolumeSpecName: "istio-data") pod "8e75a9e3-f08c-48a1-90e6-621cdcc99655" (UID: "8e75a9e3-f08c-48a1-90e6-621cdcc99655"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:33.963642 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963612    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8e75a9e3-f08c-48a1-90e6-621cdcc99655" (UID: "8e75a9e3-f08c-48a1-90e6-621cdcc99655"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:33.963642 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963618    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:33.964012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963655    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-workload-certs\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:33.964012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963670    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-data\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:33.964012 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.963668    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8e75a9e3-f08c-48a1-90e6-621cdcc99655" (UID: "8e75a9e3-f08c-48a1-90e6-621cdcc99655"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 02 11:37:33.965618 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.965583    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8e75a9e3-f08c-48a1-90e6-621cdcc99655" (UID: "8e75a9e3-f08c-48a1-90e6-621cdcc99655"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 02 11:37:33.965733 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.965623    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-kube-api-access-42jmw" (OuterVolumeSpecName: "kube-api-access-42jmw") pod "8e75a9e3-f08c-48a1-90e6-621cdcc99655" (UID: "8e75a9e3-f08c-48a1-90e6-621cdcc99655"). InnerVolumeSpecName "kube-api-access-42jmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:37:33.965733 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.965646    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-token" (OuterVolumeSpecName: "istio-token") pod "8e75a9e3-f08c-48a1-90e6-621cdcc99655" (UID: "8e75a9e3-f08c-48a1-90e6-621cdcc99655"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 02 11:37:33.965884 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:33.965870    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8e75a9e3-f08c-48a1-90e6-621cdcc99655" (UID: "8e75a9e3-f08c-48a1-90e6-621cdcc99655"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 02 11:37:34.064584 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.064543    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-envoy\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:34.064584 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.064573    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:34.064584 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.064586    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42jmw\" (UniqueName: \"kubernetes.io/projected/8e75a9e3-f08c-48a1-90e6-621cdcc99655-kube-api-access-42jmw\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:34.064826 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.064600    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istio-podinfo\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:34.064826 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.064611    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e75a9e3-f08c-48a1-90e6-621cdcc99655-istiod-ca-cert\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:34.064826 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.064622    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e75a9e3-f08c-48a1-90e6-621cdcc99655-credential-socket\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Aug 02 11:37:34.849965 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.849931    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"
Aug 02 11:37:34.850410 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.849959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv" event={"ID":"8e75a9e3-f08c-48a1-90e6-621cdcc99655","Type":"ContainerDied","Data":"6417ad03bb6544be9d6ce3332e5d1ac8379d91f3fe38156d97efbd116dd8eca1"}
Aug 02 11:37:34.850410 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.850039    2576 scope.go:117] "RemoveContainer" containerID="f13ce6b7a81d3f6da7672528eac8efaa9ab9f9e5268dacca7c920ae698eafa85"
Aug 02 11:37:34.872115 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.872090    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"]
Aug 02 11:37:34.876937 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.876916    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-3951f6-openshift-default-795fc46744-tjfdv"]
Aug 02 11:37:34.950494 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:34.950469    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" path="/var/lib/kubelet/pods/8e75a9e3-f08c-48a1-90e6-621cdcc99655/volumes"
Aug 02 11:37:42.258490 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:42.258413    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zjsm4_95918fe0-635a-4dc2-b18b-755b09b7def8/manager/0.log"
Aug 02 11:37:42.368784 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:42.368752    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-r7smk_e36da917-d50e-437a-b1f7-39b9b85ab068/manager/0.log"
Aug 02 11:37:42.484816 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:42.484788    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57f59cbf46-tsjxw_a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65/manager/0.log"
Aug 02 11:37:42.714420 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:42.714390    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kjwhq_c524b7ee-dc69-4b6c-8b06-deacc3c269e8/manager/0.log"
Aug 02 11:37:43.053163 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:43.053093    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6ffbd656-wxq55_558eb470-b32a-49a3-84c3-0d8b0d88a55b/copy-manifests/0.log"
Aug 02 11:37:43.058866 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:43.058847    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6ffbd656-wxq55_558eb470-b32a-49a3-84c3-0d8b0d88a55b/manager/0.log"
Aug 02 11:37:43.388930 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:43.388857    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-988754455-jhkss_c719e60a-3c18-4055-8dea-380a13657702/maas-api/0.log"
Aug 02 11:37:43.494941 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:43.494912    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-kf5gx_5c54d517-6a51-4953-abc7-23c5f605b83c/postgres/0.log"
Aug 02 11:37:44.351313 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:44.351279    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-5t8k2_4da1e27b-2768-4a33-a326-ddd5a661f677/manager/0.log"
Aug 02 11:37:44.455849 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:44.455817    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-w5dzt_6887945d-dd8d-4115-b9be-1b6c83c5a831/manager/0.log"
Aug 02 11:37:44.557705 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:44.557680    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-k9fpw_b7c9051c-65eb-4d50-b977-844178c1ec5b/kuadrant-console-plugin/0.log"
Aug 02 11:37:44.672003 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:44.671982    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-w26zf_6a2dc4ef-5878-419a-8138-1bb069901ef0/manager/0.log"
Aug 02 11:37:44.888168 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:44.888135    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-z74s4_f6d40d28-5cb5-43cf-b045-eb1cae85ab8e/manager/0.log"
Aug 02 11:37:45.444290 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:45.444261    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c47d49594-kbvx6_48f3f2d6-cd3d-488a-93f8-3161686612f9/kube-auth-proxy/0.log"
Aug 02 11:37:45.667347 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:45.667313    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnqf6_8b5b18a4-54de-4def-ae35-0cf0dab8deae/istio-proxy/1.log"
Aug 02 11:37:45.784801 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:45.784718    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sgwqw_fc491347-ccc2-4e2f-8d7f-a716008c5279/payload-pre-processing/0.log"
Aug 02 11:37:45.897248 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:45.897217    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w6bc7_1787e6d1-3ebf-4824-9540-88d0b2022fd6/payload-processing/0.log"
Aug 02 11:37:46.008074 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:46.008017    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b498cf56b-948pc_f663257d-867f-4ff1-8660-9ea9584d555e/router/0.log"
Aug 02 11:37:46.445344 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:46.445315    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-b86l7_0a081475-63ee-4ec9-9e90-26743e0b15ca/main/0.log"
Aug 02 11:37:46.767985 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:46.767900    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-p9wdk_1387258c-8616-4cfd-a6a8-c805b461c20c/main/0.log"
Aug 02 11:37:54.733212 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:54.733180    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xfdxj_e17cf15b-0d83-4f4d-afa8-23a6c52838f8/global-pull-secret-syncer/0.log"
Aug 02 11:37:54.832438 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:54.832396    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5l64t_4fc2ea2a-24bc-45d0-9953-b121d550ae1a/konnectivity-agent/0.log"
Aug 02 11:37:54.912953 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:37:54.912925    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-193.ec2.internal_6d6903f32483272d8f8c0d53924b7252/haproxy/0.log"
Aug 02 11:38:00.264188 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:00.264156    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-5t8k2_4da1e27b-2768-4a33-a326-ddd5a661f677/manager/0.log"
Aug 02 11:38:00.292054 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:00.292001    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-w5dzt_6887945d-dd8d-4115-b9be-1b6c83c5a831/manager/0.log"
Aug 02 11:38:00.317153 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:00.317126    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-k9fpw_b7c9051c-65eb-4d50-b977-844178c1ec5b/kuadrant-console-plugin/0.log"
Aug 02 11:38:00.415996 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:00.415956    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-w26zf_6a2dc4ef-5878-419a-8138-1bb069901ef0/manager/0.log"
Aug 02 11:38:00.522878 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:00.522796    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-z74s4_f6d40d28-5cb5-43cf-b045-eb1cae85ab8e/manager/0.log"
Aug 02 11:38:02.392150 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:02.392120    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7qz8m_12dbc719-95c7-43a9-aaf6-ed8b42db093f/node-exporter/0.log"
Aug 02 11:38:02.417463 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:02.417442    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7qz8m_12dbc719-95c7-43a9-aaf6-ed8b42db093f/kube-rbac-proxy/0.log"
Aug 02 11:38:02.435308 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:02.435285    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7qz8m_12dbc719-95c7-43a9-aaf6-ed8b42db093f/init-textfile/0.log"
Aug 02 11:38:02.863740 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:02.863709    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-hhd8z_b61d89c3-54aa-4f47-aeb6-e45095114e7b/prometheus-operator/0.log"
Aug 02 11:38:02.884968 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:02.884936    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-hhd8z_b61d89c3-54aa-4f47-aeb6-e45095114e7b/kube-rbac-proxy/0.log"
Aug 02 11:38:03.027553 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:03.027524    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d7484547-hjx52_826c0237-d299-46e3-b73d-2f7c0f3d7d15/thanos-query/0.log"
Aug 02 11:38:03.058833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:03.058802    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d7484547-hjx52_826c0237-d299-46e3-b73d-2f7c0f3d7d15/kube-rbac-proxy-web/0.log"
Aug 02 11:38:03.080638 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:03.080615    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d7484547-hjx52_826c0237-d299-46e3-b73d-2f7c0f3d7d15/kube-rbac-proxy/0.log"
Aug 02 11:38:03.101819 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:03.101792    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d7484547-hjx52_826c0237-d299-46e3-b73d-2f7c0f3d7d15/prom-label-proxy/0.log"
Aug 02 11:38:03.140121 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:03.140044    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d7484547-hjx52_826c0237-d299-46e3-b73d-2f7c0f3d7d15/kube-rbac-proxy-rules/0.log"
Aug 02 11:38:03.164610 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:03.164584    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64d7484547-hjx52_826c0237-d299-46e3-b73d-2f7c0f3d7d15/kube-rbac-proxy-metrics/0.log"
Aug 02 11:38:05.180435 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180400    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"]
Aug 02 11:38:05.180833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180780    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d1e3b6-d262-46a6-b834-e6288230e9b4" containerName="maas-api"
Aug 02 11:38:05.180833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180790    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d1e3b6-d262-46a6-b834-e6288230e9b4" containerName="maas-api"
Aug 02 11:38:05.180833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180800    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy"
Aug 02 11:38:05.180833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180806    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy"
Aug 02 11:38:05.180833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180813    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6741eb5-0db7-46e0-b2e3-00aaac63aede" containerName="maas-api"
Aug 02 11:38:05.180833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180819    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6741eb5-0db7-46e0-b2e3-00aaac63aede" containerName="maas-api"
Aug 02 11:38:05.180833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180826    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d553ebcb-9210-4414-b3a3-407442344309" containerName="maas-api"
Aug 02 11:38:05.180833 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180831    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d553ebcb-9210-4414-b3a3-407442344309" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180843    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180848    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180856    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180861    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180871    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180877    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180886    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e1f4043-7053-4c54-83f3-b91a7c4befa2" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180891    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e1f4043-7053-4c54-83f3-b91a7c4befa2" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180963    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6741eb5-0db7-46e0-b2e3-00aaac63aede" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180971    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e1f4043-7053-4c54-83f3-b91a7c4befa2" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180978    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2d1e3b6-d262-46a6-b834-e6288230e9b4" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180983    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7b9a0ec-da4d-4d23-b8ce-a4fa12f97659" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180992    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.180997    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.181003    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a92d6d-e10d-4791-b04b-a5732dc701f9" containerName="istio-proxy"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.181009    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d553ebcb-9210-4414-b3a3-407442344309" containerName="maas-api"
Aug 02 11:38:05.181164 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.181013    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e75a9e3-f08c-48a1-90e6-621cdcc99655" containerName="istio-proxy"
Aug 02 11:38:05.184005 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.183991    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.186890 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.186862    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9p2g8\"/\"openshift-service-ca.crt\""
Aug 02 11:38:05.187936 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.187915    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9p2g8\"/\"kube-root-ca.crt\""
Aug 02 11:38:05.188099 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.187915    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9p2g8\"/\"default-dockercfg-jktht\""
Aug 02 11:38:05.191718 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.191387    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"]
Aug 02 11:38:05.333931 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.333901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-proc\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.334121 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.333936    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrbsj\" (UniqueName: \"kubernetes.io/projected/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-kube-api-access-rrbsj\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.334121 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.334012    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-podres\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.334121 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.334087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-lib-modules\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.334239 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.334169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-sys\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.434851 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.434778    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-proc\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.434851 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.434817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrbsj\" (UniqueName: \"kubernetes.io/projected/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-kube-api-access-rrbsj\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.435066 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.434872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-podres\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.435066 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.434908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-proc\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.435066 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.434917    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-lib-modules\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.435066 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.434978    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-sys\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.435066 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.435057    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-lib-modules\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.435282 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.435057    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-podres\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.435282 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.435105    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-sys\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.443003 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.442977    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrbsj\" (UniqueName: \"kubernetes.io/projected/5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc-kube-api-access-rrbsj\") pod \"perf-node-gather-daemonset-4mx4m\" (UID: \"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc\") " pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.495180 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.495154    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:05.631335 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.631308    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"]
Aug 02 11:38:05.634461 ip-10-0-135-193 kubenswrapper[2576]: W0802 11:38:05.634436    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5a2cfaca_5fea_4a95_9222_9af4d5f6e4cc.slice/crio-d14ca45f6d738319396fedc91f75a9970f8110898fb756c963defb7bd457f34d WatchSource:0}: Error finding container d14ca45f6d738319396fedc91f75a9970f8110898fb756c963defb7bd457f34d: Status 404 returned error can't find the container with id d14ca45f6d738319396fedc91f75a9970f8110898fb756c963defb7bd457f34d
Aug 02 11:38:05.644991 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:05.644971    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7d654958d9-4k68m_787b9dfa-1cf1-4ec5-80fd-b63982742c1e/console/0.log"
Aug 02 11:38:06.033521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:06.033436    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m" event={"ID":"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc","Type":"ContainerStarted","Data":"e6c27fb720563df4880f6c9bbd6059e9064a69c49d2ab8fdf4d2329d75235a4c"}
Aug 02 11:38:06.033521 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:06.033482    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m" event={"ID":"5a2cfaca-5fea-4a95-9222-9af4d5f6e4cc","Type":"ContainerStarted","Data":"d14ca45f6d738319396fedc91f75a9970f8110898fb756c963defb7bd457f34d"}
Aug 02 11:38:06.033696 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:06.033541    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:06.050734 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:06.050689    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m" podStartSLOduration=1.05067524 podStartE2EDuration="1.05067524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-02 11:38:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-02 11:38:06.048146493 +0000 UTC m=+3887.578729121" watchObservedRunningTime="2026-08-02 11:38:06.05067524 +0000 UTC m=+3887.581257868"
Aug 02 11:38:06.382010 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:06.381938    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-l5dsq_f63fbd96-806b-4a29-8c20-77f857e4c36d/volume-data-source-validator/0.log"
Aug 02 11:38:07.454167 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:07.454139    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6hstf_0d7a5405-d4e4-40b8-90c9-06f6b9160b12/dns/0.log"
Aug 02 11:38:07.493962 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:07.493942    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6hstf_0d7a5405-d4e4-40b8-90c9-06f6b9160b12/kube-rbac-proxy/0.log"
Aug 02 11:38:07.725912 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:07.725840    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkcz8_a3cfe13b-9aba-4b1f-8e83-33f2f2d7f786/dns-node-resolver/0.log"
Aug 02 11:38:08.340251 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:08.340223    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fbkmq_92f58dc5-7223-40a2-85bb-020e1ef6d6a1/node-ca/0.log"
Aug 02 11:38:09.489193 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:09.489163    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c47d49594-kbvx6_48f3f2d6-cd3d-488a-93f8-3161686612f9/kube-auth-proxy/0.log"
Aug 02 11:38:09.543040 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:09.543012    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnqf6_8b5b18a4-54de-4def-ae35-0cf0dab8deae/istio-proxy/0.log"
Aug 02 11:38:09.549295 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:09.549276    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnqf6_8b5b18a4-54de-4def-ae35-0cf0dab8deae/istio-proxy/1.log"
Aug 02 11:38:09.579983 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:09.579959    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sgwqw_fc491347-ccc2-4e2f-8d7f-a716008c5279/payload-pre-processing/0.log"
Aug 02 11:38:09.612934 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:09.612915    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w6bc7_1787e6d1-3ebf-4824-9540-88d0b2022fd6/payload-processing/0.log"
Aug 02 11:38:09.633520 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:09.633500    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b498cf56b-948pc_f663257d-867f-4ff1-8660-9ea9584d555e/router/0.log"
Aug 02 11:38:10.440378 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:10.440341    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2t452_9cab4866-7565-40c3-bbc3-c3c7d2e9656a/serve-healthcheck-canary/0.log"
Aug 02 11:38:11.138320 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:11.138289    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gqnd5_a662df23-b5df-4447-b0bf-713a62e1770a/kube-rbac-proxy/0.log"
Aug 02 11:38:11.160953 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:11.160926    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gqnd5_a662df23-b5df-4447-b0bf-713a62e1770a/exporter/0.log"
Aug 02 11:38:11.181781 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:11.181758    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gqnd5_a662df23-b5df-4447-b0bf-713a62e1770a/extractor/0.log"
Aug 02 11:38:12.048899 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:12.048866    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9p2g8/perf-node-gather-daemonset-4mx4m"
Aug 02 11:38:13.335289 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:13.335261    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zjsm4_95918fe0-635a-4dc2-b18b-755b09b7def8/manager/0.log"
Aug 02 11:38:13.549830 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:13.549801    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-r7smk_e36da917-d50e-437a-b1f7-39b9b85ab068/manager/0.log"
Aug 02 11:38:13.711910 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:13.711879    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57f59cbf46-tsjxw_a8db5efb-5a93-4fe6-a2bc-7a1cb6526a65/manager/0.log"
Aug 02 11:38:13.806004 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:13.805983    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kjwhq_c524b7ee-dc69-4b6c-8b06-deacc3c269e8/manager/0.log"
Aug 02 11:38:13.991803 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:13.991730    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6ffbd656-wxq55_558eb470-b32a-49a3-84c3-0d8b0d88a55b/manager/0.log"
Aug 02 11:38:14.011642 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:14.011618    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6ffbd656-wxq55_558eb470-b32a-49a3-84c3-0d8b0d88a55b/copy-manifests/0.log"
Aug 02 11:38:16.425021 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:16.424948    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c47d49594-kbvx6_48f3f2d6-cd3d-488a-93f8-3161686612f9/kube-auth-proxy/0.log"
Aug 02 11:38:16.468811 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:16.468785    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnqf6_8b5b18a4-54de-4def-ae35-0cf0dab8deae/istio-proxy/0.log"
Aug 02 11:38:16.474762 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:16.474746    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnqf6_8b5b18a4-54de-4def-ae35-0cf0dab8deae/istio-proxy/1.log"
Aug 02 11:38:16.494568 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:16.494543    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sgwqw_fc491347-ccc2-4e2f-8d7f-a716008c5279/payload-pre-processing/0.log"
Aug 02 11:38:16.515310 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:16.515284    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w6bc7_1787e6d1-3ebf-4824-9540-88d0b2022fd6/payload-processing/0.log"
Aug 02 11:38:16.530757 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:16.530734    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b498cf56b-948pc_f663257d-867f-4ff1-8660-9ea9584d555e/router/0.log"
Aug 02 11:38:18.136157 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:18.136126    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c47d49594-kbvx6_48f3f2d6-cd3d-488a-93f8-3161686612f9/kube-auth-proxy/0.log"
Aug 02 11:38:18.172935 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:18.172906    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnqf6_8b5b18a4-54de-4def-ae35-0cf0dab8deae/istio-proxy/0.log"
Aug 02 11:38:18.179292 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:18.179263    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dnqf6_8b5b18a4-54de-4def-ae35-0cf0dab8deae/istio-proxy/1.log"
Aug 02 11:38:18.204290 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:18.204262    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sgwqw_fc491347-ccc2-4e2f-8d7f-a716008c5279/payload-pre-processing/0.log"
Aug 02 11:38:18.225761 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:18.225734    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w6bc7_1787e6d1-3ebf-4824-9540-88d0b2022fd6/payload-processing/0.log"
Aug 02 11:38:18.243196 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:18.243170    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b498cf56b-948pc_f663257d-867f-4ff1-8660-9ea9584d555e/router/0.log"
Aug 02 11:38:19.274989 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:19.274965    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:38:19.310367 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:19.310336    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j98mk_4a55c89e-0c43-4f48-be56-546f81bdc357/ovn-acl-logging/0.log"
Aug 02 11:38:21.830498 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:21.830460    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-59tjn_5c7e82f0-5926-4b13-9dba-ccafb2b13278/kube-multus/0.log"
Aug 02 11:38:21.885004 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:21.884973    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rj9b_a2fed233-9b0d-44ad-9778-3ec67dd03656/kube-multus-additional-cni-plugins/0.log"
Aug 02 11:38:21.904777 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:21.904755    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rj9b_a2fed233-9b0d-44ad-9778-3ec67dd03656/egress-router-binary-copy/0.log"
Aug 02 11:38:21.924688 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:21.924664    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rj9b_a2fed233-9b0d-44ad-9778-3ec67dd03656/cni-plugins/0.log"
Aug 02 11:38:21.944428 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:21.944408    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rj9b_a2fed233-9b0d-44ad-9778-3ec67dd03656/bond-cni-plugin/0.log"
Aug 02 11:38:21.964548 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:21.964530    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rj9b_a2fed233-9b0d-44ad-9778-3ec67dd03656/routeoverride-cni/0.log"
Aug 02 11:38:21.984536 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:21.984505    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rj9b_a2fed233-9b0d-44ad-9778-3ec67dd03656/whereabouts-cni-bincopy/0.log"
Aug 02 11:38:22.004680 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:22.004662    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rj9b_a2fed233-9b0d-44ad-9778-3ec67dd03656/whereabouts-cni/0.log"
Aug 02 11:38:22.418400 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:22.418374    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mmf2n_7f208a37-3b97-4072-ac20-9ffbf8732d84/network-metrics-daemon/0.log"
Aug 02 11:38:22.437633 ip-10-0-135-193 kubenswrapper[2576]: I0802 11:38:22.437611    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mmf2n_7f208a37-3b97-4072-ac20-9ffbf8732d84/kube-rbac-proxy/0.log"