Sep 18 15:32:19.871806 ip-10-0-130-208 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 15:32:19.871815 ip-10-0-130-208 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 15:32:19.871823 ip-10-0-130-208 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 15:32:19.872068 ip-10-0-130-208 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 15:32:29.911511 ip-10-0-130-208 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 15:32:29.911529 ip-10-0-130-208 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ee324b5ceba6469bbb10df02e72dd9e4 --
Sep 18 15:34:33.775201 ip-10-0-130-208 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 15:34:37.272299 ip-10-0-130-208 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 15:34:37.272299 ip-10-0-130-208 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 15:34:37.272299 ip-10-0-130-208 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 15:34:37.272299 ip-10-0-130-208 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 15:34:37.272299 ip-10-0-130-208 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 15:34:37.273053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.272380    2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 15:34:37.276381 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276364    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 15:34:37.276381 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276380    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276383    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276386    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276389    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276392    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276395    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276398    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276401    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276403    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276406    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276409    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276411    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276414    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276417    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276419    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276422    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276424    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276427    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276429    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276431    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 15:34:37.276443 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276435    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276438    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276441    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276456    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276459    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276461    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276464    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276467    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276469    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276472    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276475    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276477    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276480    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276483    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276486    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276488    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276491    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276494    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276496    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276498    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 15:34:37.276995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276501    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276503    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276506    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276508    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276511    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276513    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276515    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276519    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276523    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276525    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276528    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276532    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276534    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276537    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276542    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276544    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276547    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276549    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276553    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 15:34:37.277518 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276557    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276561    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276564    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276567    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276570    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276573    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276576    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276578    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276581    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276584    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276586    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276590    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276593    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276595    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276598    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276602    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276604    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276607    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276609    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 15:34:37.278004 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276612    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276614    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276617    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276619    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276621    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276624    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276626    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276629    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276632    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276635    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276637    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276640    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276643    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276645    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276648    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276651    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276653    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276656    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276658    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 15:34:37.278454 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.276660    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277401    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277410    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277413    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277417    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277419    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277423    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277426    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277429    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277431    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277434    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277436    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277439    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277442    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277444    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277447    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277449    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277452    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277454    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277457    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 15:34:37.278922 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277460    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277464    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277467    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277470    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277472    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277474    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277477    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277480    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277483    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277485    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277488    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277491    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277493    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277496    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277498    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277501    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277503    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277506    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277508    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 15:34:37.279415 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277512    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277514    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277517    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277519    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277522    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277525    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277527    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277530    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277535    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277538    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277541    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277545    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277548    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277551    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277554    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277557    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277559    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277562    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277564    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 15:34:37.279893 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277567    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277569    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277572    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277575    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277577    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277580    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277582    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277585    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277587    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277590    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277592    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277594    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277598    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277600    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277603    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277606    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277609    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277611    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277614    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277616    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 15:34:37.280353 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277619    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277622    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277625    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277627    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277630    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277633    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277635    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277638    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277640    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277643    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277645    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277648    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277650    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277653    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277655    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277657    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277660    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277663    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277665    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277667    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 15:34:37.280847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277670    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.277674    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278062    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278068    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278073    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278077    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278081    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278084    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278087    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278090    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278093    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278096    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278098    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278101    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278104    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278106    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278108    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278111    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278114    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 15:34:37.281351 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278117    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278120    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278123    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278125    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278129    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278133    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278137    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278140    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278143    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278146    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278149    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278152    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278163    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278166    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278169    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278171    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278174    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278177    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278180    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 15:34:37.281856 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278183    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278185    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278188    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278191    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278193    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278196    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278198    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278201    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278204    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278206    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278209    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278214    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278216    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278220    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278223    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278225    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278228    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278231    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278233    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278236    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 15:34:37.282369 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278238    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278241    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278243    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278246    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278248    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278251    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278254    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278256    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278259    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278262    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278265    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278267    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278270    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278272    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278275    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278278    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278282    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278285    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278288    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278291    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 15:34:37.282859 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278294    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278296    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278299    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278305    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278308    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278310    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278313    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278316    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278318    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278321    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278324    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278326    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278329    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278331    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278334    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278336    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278339    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278341    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278344    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278347    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 15:34:37.283362 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278349    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278352    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.278355    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278427    2585 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278462    2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278470    2585 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278475    2585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278481    2585 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278486    2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278490    2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278494    2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278497    2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278500    2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278503    2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278506    2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278511    2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278514    2585 flags.go:64] FLAG: --cgroup-root=""
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278517    2585 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278519    2585 flags.go:64] FLAG: --client-ca-file=""
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278522    2585 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278525    2585 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278529    2585 flags.go:64] FLAG: --cluster-domain=""
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278532    2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278536    2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 15:34:37.283833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278539    2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278543    2585 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278546    2585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278550    2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278553    2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278556    2585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278559    2585 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278562    2585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278565    2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278569    2585 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278572    2585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278576    2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278579    2585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278582    2585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278585    2585 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278588    2585 flags.go:64] FLAG: --enable-server="true"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278592    2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278597    2585 flags.go:64] FLAG: --event-burst="100"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278600    2585 flags.go:64] FLAG: --event-qps="50"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278603    2585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278606    2585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278609    2585 flags.go:64] FLAG: --eviction-hard=""
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278613    2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278617    2585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278621    2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 15:34:37.284428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278624    2585 flags.go:64] FLAG: --eviction-soft=""
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278626    2585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278629    2585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278632    2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278635    2585 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278638    2585 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278641    2585 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278643    2585 flags.go:64] FLAG: --feature-gates=""
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278647    2585 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278650    2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278654    2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278657    2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278660    2585 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278662    2585 flags.go:64] FLAG: --help="false"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278666    2585 flags.go:64] FLAG: --hostname-override="ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278668    2585 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278673    2585 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278676    2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278679    2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278682    2585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278685    2585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278688    2585 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278690    2585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 15:34:37.285053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278693    2585 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278696    2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278700    2585 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278702    2585 flags.go:64] FLAG: --kube-reserved=""
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278705    2585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278708    2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278711    2585 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278716    2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278719    2585 flags.go:64] FLAG: --lock-file=""
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278722    2585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278724    2585 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278728    2585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278733    2585 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278736    2585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278738    2585 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278741    2585 flags.go:64] FLAG: --logging-format="text"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278744    2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278747    2585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278750    2585 flags.go:64] FLAG: --manifest-url=""
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278753    2585 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278757    2585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278760    2585 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278764    2585 flags.go:64] FLAG: --max-pods="110"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278767    2585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278770    2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 15:34:37.285705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278774    2585 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278777    2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278780    2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278782    2585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278785    2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278792    2585 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278795    2585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278798    2585 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278801    2585 flags.go:64] FLAG: --pod-cidr=""
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278804    2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278809    2585 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278812    2585 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278815    2585 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278818    2585 flags.go:64] FLAG: --port="10250"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278821    2585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278825    2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b4108eaacf25c9fa"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278828    2585 flags.go:64] FLAG: --qos-reserved=""
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278831    2585 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278848    2585 flags.go:64] FLAG: --register-node="true"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278851    2585 flags.go:64] FLAG: --register-with-taints=""
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278855    2585 flags.go:64] FLAG: --registry-burst="10"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278858    2585 flags.go:64] FLAG: --registry-qps="5"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278861    2585 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278864    2585 flags.go:64] FLAG: --reserved-memory=""
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278867    2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 15:34:37.286337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278870    2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278873    2585 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278876    2585 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278879    2585 flags.go:64] FLAG: --runonce="false"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278882    2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278885    2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278888    2585 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278892    2585 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278895    2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278898    2585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278901    2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278904    2585 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278907    2585 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278910    2585 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278912    2585 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278915    2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278918    2585 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278922    2585 flags.go:64] FLAG: --system-cgroups=""
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278925    2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278930    2585 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278933    2585 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278936    2585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278940    2585 flags.go:64] FLAG: --tls-min-version=""
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278943    2585 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278946    2585 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 15:34:37.286990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278949    2585 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278951    2585 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278954    2585 flags.go:64] FLAG: --v="2"
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278958    2585 flags.go:64] FLAG: --version="false"
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278962    2585 flags.go:64] FLAG: --vmodule=""
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278965    2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.278969    2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279090    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279093    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279096    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279099    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279102    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279104    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279107    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279111    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279113    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279116    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279118    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279121    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279124    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279126    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 15:34:37.287598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279129    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279132    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279134    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279137    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279145    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279148    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279150    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279155    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279158    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279161    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279165    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279168    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279170    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279173    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279176    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279179    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279182    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279185    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279187    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 15:34:37.288124 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279190    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279192    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279195    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279197    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279200    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279202    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279206    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279209    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279212    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279214    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279217    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279219    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279222    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279224    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279227    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279229    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279232    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279235    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279251    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279254    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 15:34:37.288587 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279258    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279262    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279265    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279268    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279272    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279276    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279278    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279281    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279284    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279287    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279290    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279292    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279295    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279297    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279300    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279302    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279305    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279307    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279311    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279313    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 15:34:37.289246 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279316    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279319    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279321    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279324    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279326    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279329    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279331    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279334    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279337    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279339    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279342    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279350    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279353    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279356    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279358    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279361    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279363    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279366    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279369    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279371    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 15:34:37.290058 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279374    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279376    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279379    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279381    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279384    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.279386    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.279391    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.286944    2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.286959    2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.286979    2585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.286985    2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287052    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287057    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287060    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287064    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 15:34:37.290580 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287067    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287070    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287073    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287076    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287079    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287082    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287085    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287089    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287092    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287096    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287099    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287102    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287104    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287107    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287110    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287113    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287115    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287118    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287121    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 15:34:37.290982 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287123    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287127    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287131    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287134    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287137    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287140    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287144    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287147    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287149    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287152    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287155    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287157    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287160    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287164    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287167    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287170    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287173    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287176    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287179    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 15:34:37.291439 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287182    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287185    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287187    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287190    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287192    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287195    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287197    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287200    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287203    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287205    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287208    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287210    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287213    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287216    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287218    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287221    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287223    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287226    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287228    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287232    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 15:34:37.291995 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287235    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287238    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287241    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287243    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287246    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287248    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287251    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287253    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287256    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287259    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287261    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287264    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287267    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287269    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287272    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287274    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287277    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287279    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287282    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287285    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 15:34:37.292490 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287287    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287290    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287292    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287295    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287298    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287300    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287303    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287306    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287308    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287311    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287313    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287317    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287320    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287323    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287325    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287328    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 15:34:37.292998 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287331    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.287335    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287466    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287471    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287474    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287477    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287479    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287482    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287485    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287487    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287490    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287492    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287495    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287498    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 15:34:37.293425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287500    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287503    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287505    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287508    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287511    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287513    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287516    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287518    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287521    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287523    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287526    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287528    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287531    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287533    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287536    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287538    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287541    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287544    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287548    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287552    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 15:34:37.293767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287555    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287558    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287560    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287563    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287565    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287568    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287571    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287573    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287576    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287579    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287583    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287586    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287589    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287591    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287594    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287596    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287599    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287602    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287604    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287607    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 15:34:37.294295 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287610    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287612    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287615    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287618    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287621    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287624    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287627    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287630    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287632    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287635    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287637    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287640    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287643    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287645    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287648    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287650    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287653    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287655    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287658    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287661    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 15:34:37.294789 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287663    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287665    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287668    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287671    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287673    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287676    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287678    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287681    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287684    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287686    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287689    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287691    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287694    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287696    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287699    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287701    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287704    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287706    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287709    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287711    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 15:34:37.295330 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287714    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287716    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287719    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287722    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287725    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287727    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:37.287730    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.287735    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.287849    2585 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.291496    2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.292627    2585 server.go:1019] "Starting client certificate rotation"
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.292727    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.292765    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 15:34:37.295845 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.295324    2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 15:34:37.296417 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.296400    2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 15:34:37.302437 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.302421    2585 log.go:25] "Validated CRI v1 runtime API"
Sep 18 15:34:37.303939 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.303927    2585 log.go:25] "Validated CRI v1 image API"
Sep 18 15:34:37.304595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.304585    2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 15:34:37.307870 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.307833    2585 fs.go:135] Filesystem UUIDs: map[55441eab-da65-4f0f-8e6f-e0e7bccbb7b4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 83d52d0e-432f-49a5-b731-ba36a2721421:/dev/nvme0n1p3]
Sep 18 15:34:37.307933 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.307869    2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 15:34:37.313479 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.313344    2585 manager.go:216] Machine: {Timestamp:2026-09-18 15:34:37.312883096 +0000 UTC m=+1.971675745 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3148013 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2389a06a9dd33bb547e891071b4bd0 SystemUUID:ec2389a0-6a9d-d33b-b547-e891071b4bd0 BootID:ee324b5c-eba6-469b-bb10-df02e72dd9e4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d7:12:87:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d7:12:87:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:26:fd:cb:36:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 15:34:37.313479 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.313470    2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 15:34:37.313607 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.313540    2585 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 15:34:37.315227 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.315197    2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 15:34:37.315397 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.315235    2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-208.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 15:34:37.316542 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.316529    2585 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 15:34:37.316607 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.316547    2585 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 15:34:37.316607 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.316559    2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 15:34:37.316607 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.316574    2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 15:34:37.316745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.316655    2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 15:34:37.319628 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.319614    2585 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 15:34:37.319752 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.319742    2585 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 15:34:37.319806 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.319780    2585 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 15:34:37.319806 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.319801    2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 15:34:37.319914 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.319822    2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 15:34:37.319914 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.319852    2585 kubelet.go:398] "Adding apiserver pod source"
Sep 18 15:34:37.319914 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.319865    2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 15:34:37.320236 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.320209    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 15:34:37.320327 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.320249    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 15:34:37.324804 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.324785    2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 15:34:37.327089 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.327072    2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 15:34:37.327179 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.327110    2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 15:34:37.328197 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328179    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 15:34:37.328275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328205    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 15:34:37.328275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328214    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 15:34:37.328275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328223    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 15:34:37.328275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328236    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 15:34:37.328275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328244    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 15:34:37.328275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328262    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 15:34:37.328275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328269    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 15:34:37.328275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328278    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 15:34:37.328486 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328287    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 15:34:37.328486 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328300    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 15:34:37.328486 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328308    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 15:34:37.328486 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328333    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 15:34:37.328486 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.328341    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 15:34:37.329849 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.329821    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 15:34:37.332143 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.332130    2585 server.go:1268] "Started kubelet"
Sep 18 15:34:37.332465 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.332441    2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 15:34:37.332679 ip-10-0-130-208 systemd[1]: Started Kubernetes Kubelet.
Sep 18 15:34:37.333746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.333696    2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 15:34:37.333887 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.333874    2585 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 15:34:37.334135 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.334117    2585 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 15:34:37.335052 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.335027    2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 15:34:37.338924 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.338906    2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-208.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 15:34:37.339016 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.339000    2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 15:34:37.339586 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.339551    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-208.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 15:34:37.339586 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.339554    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 15:34:37.340122 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.340108    2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 15:34:37.340184 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.340127    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 15:34:37.340240 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.340179    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:37.340697 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.340680    2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 15:34:37.340697 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.340680    2585 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 15:34:37.340821 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.340707    2585 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 15:34:37.340883 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.340830    2585 factory.go:55] Registering systemd factory
Sep 18 15:34:37.340883 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.340858    2585 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 15:34:37.340883 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.340866    2585 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 15:34:37.340883 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.340872    2585 factory.go:223] Registration of the systemd container factory successfully
Sep 18 15:34:37.341096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.341081    2585 factory.go:153] Registering CRI-O factory
Sep 18 15:34:37.341155 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.341104    2585 factory.go:223] Registration of the crio container factory successfully
Sep 18 15:34:37.341155 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.341152    2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 15:34:37.341248 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.341170    2585 factory.go:103] Registering Raw factory
Sep 18 15:34:37.341248 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.341185    2585 manager.go:1196] Started watching for new ooms in manager
Sep 18 15:34:37.341520 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.341508    2585 manager.go:349] Starting recovery of all containers
Sep 18 15:34:37.367360 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.366929    2585 manager.go:354] Recovery completed
Sep 18 15:34:37.368018 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.367990    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 15:34:37.368166 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.368143    2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-208.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 15:34:37.368166 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.368154    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hvljr"
Sep 18 15:34:37.368286 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.368175    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hvljr"
Sep 18 15:34:37.373099 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.373084    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 15:34:37.373644 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.372563    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-208.ec2.internal.18d6743607abdd23  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-208.ec2.internal,UID:ip-10-0-130-208.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-208.ec2.internal,},FirstTimestamp:2026-09-18 15:34:37.332110627 +0000 UTC m=+1.990903277,LastTimestamp:2026-09-18 15:34:37.332110627 +0000 UTC m=+1.990903277,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-208.ec2.internal,}"
Sep 18 15:34:37.374739 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.374724    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 15:34:37.374887 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.374754    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 15:34:37.374887 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.374768    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientPID"
Sep 18 15:34:37.375268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.375255    2585 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 15:34:37.375268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.375267    2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 15:34:37.375339 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.375285    2585 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 15:34:37.375339 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.375291    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 15:34:37.375986 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.375971    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 15:34:37.376043 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.375996    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 15:34:37.376043 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.376007    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientPID"
Sep 18 15:34:37.376538 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.376527    2585 policy_none.go:49] "None policy: Start"
Sep 18 15:34:37.376570 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.376542    2585 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 15:34:37.376570 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.376552    2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 15:34:37.377621 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.377610    2585 policy_none.go:47] "Start"
Sep 18 15:34:37.416446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.416435    2585 manager.go:337] "Starting Device Plugin manager"
Sep 18 15:34:37.424409 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.416478    2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 15:34:37.424409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.416490    2585 server.go:87] "Starting device plugin registration server"
Sep 18 15:34:37.424409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.416740    2585 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 15:34:37.424409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.416763    2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 15:34:37.424409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.416890    2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 15:34:37.424409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.416961    2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 15:34:37.424409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.416970    2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 15:34:37.424409 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.417399    2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 15:34:37.424409 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.417453    2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:37.515300 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.515271    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 15:34:37.516434 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.516420    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 15:34:37.516493 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.516466    2585 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 15:34:37.516493 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.516489    2585 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 15:34:37.516592 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.516523    2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 15:34:37.516833 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.516816    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 15:34:37.517705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.517684    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 15:34:37.517785 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.517717    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 15:34:37.517785 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.517732    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientPID"
Sep 18 15:34:37.517785 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.517760    2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.519067 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.519050    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 15:34:37.526050 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.526015    2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.526050 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.526036    2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-208.ec2.internal\": node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:37.543770 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.543753    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:37.617561 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.617531    2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal"]
Sep 18 15:34:37.617680 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.617603    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 15:34:37.619142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.619125    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 15:34:37.619199 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.619163    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 15:34:37.619199 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.619177    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientPID"
Sep 18 15:34:37.620502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.620490    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 15:34:37.620630 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.620617    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.620673 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.620646    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 15:34:37.621198 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.621182    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 15:34:37.621289 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.621211    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 15:34:37.621289 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.621225    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientPID"
Sep 18 15:34:37.621289 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.621191    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 15:34:37.621289 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.621280    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 15:34:37.621289 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.621291    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientPID"
Sep 18 15:34:37.622565 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.622550    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.622645 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.622577    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 15:34:37.623199 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.623185    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 15:34:37.623290 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.623209    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 15:34:37.623290 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.623222    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeHasSufficientPID"
Sep 18 15:34:37.642034 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.642018    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7aedb2ec9046cdbc367d9c6b6d51d18-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal\" (UID: \"f7aedb2ec9046cdbc367d9c6b6d51d18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.642128 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.642041    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7aedb2ec9046cdbc367d9c6b6d51d18-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal\" (UID: \"f7aedb2ec9046cdbc367d9c6b6d51d18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.642128 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.642056    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aa0a25f1f0b432907b52509419f94e73-config\") pod \"kube-apiserver-proxy-ip-10-0-130-208.ec2.internal\" (UID: \"aa0a25f1f0b432907b52509419f94e73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.644095 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.644071    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:37.651366 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.651352    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-208.ec2.internal\" not found" node="ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.655796 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.655777    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-208.ec2.internal\" not found" node="ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.742216 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.742197    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aa0a25f1f0b432907b52509419f94e73-config\") pod \"kube-apiserver-proxy-ip-10-0-130-208.ec2.internal\" (UID: \"aa0a25f1f0b432907b52509419f94e73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.742335 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.742223    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7aedb2ec9046cdbc367d9c6b6d51d18-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal\" (UID: \"f7aedb2ec9046cdbc367d9c6b6d51d18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.742335 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.742239    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7aedb2ec9046cdbc367d9c6b6d51d18-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal\" (UID: \"f7aedb2ec9046cdbc367d9c6b6d51d18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.742335 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.742285    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7aedb2ec9046cdbc367d9c6b6d51d18-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal\" (UID: \"f7aedb2ec9046cdbc367d9c6b6d51d18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.742335 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.742292    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aa0a25f1f0b432907b52509419f94e73-config\") pod \"kube-apiserver-proxy-ip-10-0-130-208.ec2.internal\" (UID: \"aa0a25f1f0b432907b52509419f94e73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.742335 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.742314    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7aedb2ec9046cdbc367d9c6b6d51d18-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal\" (UID: \"f7aedb2ec9046cdbc367d9c6b6d51d18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.744163 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.744144    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:37.844912 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.844849    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:37.945348 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:37.945320    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:37.955479 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.955450    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:37.959785 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:37.959562    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:38.045805 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:38.045633    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:38.146149 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:38.146089    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:38.246589 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:38.246564    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:38.292738 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.292715    2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 15:34:38.293141 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.292870    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 15:34:38.346952 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:38.346924    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:38.373445 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.373401    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 15:29:37 +0000 UTC" deadline="2028-04-15 00:30:24.172827965 +0000 UTC"
Sep 18 15:34:38.373445 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.373440    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13784h55m45.799391686s"
Sep 18 15:34:38.447166 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:38.447108    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:38.451086 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.451062    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 15:34:38.547984 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:38.547954    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:38.602092 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:38.602056    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7aedb2ec9046cdbc367d9c6b6d51d18.slice/crio-6abfbea62de89ec71f7da450340a50d256656edb661976bb242d66220e00b68e WatchSource:0}: Error finding container 6abfbea62de89ec71f7da450340a50d256656edb661976bb242d66220e00b68e: Status 404 returned error can't find the container with id 6abfbea62de89ec71f7da450340a50d256656edb661976bb242d66220e00b68e
Sep 18 15:34:38.602456 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:38.602436    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa0a25f1f0b432907b52509419f94e73.slice/crio-9c7000f1fabf12c2b7d926514cde886a3dfd53dd3ecff93268f5f6d2bfbd7dd0 WatchSource:0}: Error finding container 9c7000f1fabf12c2b7d926514cde886a3dfd53dd3ecff93268f5f6d2bfbd7dd0: Status 404 returned error can't find the container with id 9c7000f1fabf12c2b7d926514cde886a3dfd53dd3ecff93268f5f6d2bfbd7dd0
Sep 18 15:34:38.608048 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.608032    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 15:34:38.648531 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:38.648506    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:38.723171 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.723108    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 15:34:38.748851 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:38.748814    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-208.ec2.internal\" not found"
Sep 18 15:34:38.751107 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.751091    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 15:34:38.840955 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.840931    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:38.852284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.852268    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 15:34:38.853338 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.853326    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal"
Sep 18 15:34:38.871496 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:38.871480    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 15:34:39.320263 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.320235    2585 apiserver.go:52] "Watching apiserver"
Sep 18 15:34:39.334920 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.334707    2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 15:34:39.335230 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.335202    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-4rnz2","openshift-network-diagnostics/network-check-target-qq2zh","kube-system/konnectivity-agent-dl59b","kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal","openshift-cluster-node-tuning-operator/tuned-7hmc7","openshift-image-registry/node-ca-hhm7w","openshift-network-operator/iptables-alerter-4lpkq","openshift-ovn-kubernetes/ovnkube-node-6wqf6","kube-system/global-pull-secret-syncer-bvgpr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb","openshift-dns/node-resolver-rnzfc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal","openshift-multus/multus-additional-cni-plugins-ddhnl","openshift-multus/multus-bcszs"]
Sep 18 15:34:39.337874 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.337324    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.338825 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.338807    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.341236 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.340455    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:39.341236 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.340856    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 15:34:39.342770 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.342555    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.343375 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.343356    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.344528 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.344502    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.345129 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.344770    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 15:34:39.345129 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.344976    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dhh5k\""
Sep 18 15:34:39.345129 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.344998    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 15:34:39.345994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.345918    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:39.346089 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.345991    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:39.347217 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.347186    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 15:34:39.347217 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.347202    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 15:34:39.347434 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.347418    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.348660 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.348642    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.348901    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bfldw\""
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.348955    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.348971    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349014    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-daemon-config\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349048    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-conf-dir\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349085    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-socket-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349115    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-run-netns\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349156    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-run-multus-certs\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349182    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fbfq\" (UniqueName: \"kubernetes.io/projected/846e19da-3e22-44c5-a6d5-5dd121781a03-kube-api-access-8fbfq\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349198    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349211    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349222    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-registration-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349247    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-var-lib-kubelet\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349256    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hqh4j\""
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349282    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6a044247-0dee-476f-b2e4-309e4e0982d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349201    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349307    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-system-cni-dir\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349337    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-var-lib-cni-bin\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349380    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-device-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.349442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349416    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-etc-selinux\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349441    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-sys-fs\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349454    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349467    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-cni-dir\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349490    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/846e19da-3e22-44c5-a6d5-5dd121781a03-cni-binary-copy\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.349508    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349513    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a044247-0dee-476f-b2e4-309e4e0982d8-system-cni-dir\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349550    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-cnibin\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349576    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn2wb\" (UniqueName: \"kubernetes.io/projected/070e3db3-8130-4977-843e-9d520bb22197-kube-api-access-rn2wb\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349599    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a044247-0dee-476f-b2e4-309e4e0982d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349623    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6a044247-0dee-476f-b2e4-309e4e0982d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349647    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349669    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-var-lib-cni-multus\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349691    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-etc-kubernetes\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349719    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a044247-0dee-476f-b2e4-309e4e0982d8-cnibin\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349742    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkhcr\" (UniqueName: \"kubernetes.io/projected/6a044247-0dee-476f-b2e4-309e4e0982d8-kube-api-access-zkhcr\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.350409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349766    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae988a2f-8f5c-4506-a5ba-ba8d7d79212e-agent-certs\") pod \"konnectivity-agent-dl59b\" (UID: \"ae988a2f-8f5c-4506-a5ba-ba8d7d79212e\") " pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349789    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-socket-dir-parent\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349811    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-run-k8s-cni-cncf-io\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349848    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a044247-0dee-476f-b2e4-309e4e0982d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349872    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae988a2f-8f5c-4506-a5ba-ba8d7d79212e-konnectivity-ca\") pod \"konnectivity-agent-dl59b\" (UID: \"ae988a2f-8f5c-4506-a5ba-ba8d7d79212e\") " pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349907    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349960    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.349908    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-hostroot\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.350002    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.350056    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pqbgz\""
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.350094    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.350119    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7w6b5\""
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.350151    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4t499\""
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.350648    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:39.351213 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.350722    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:39.351849 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.351758    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.353162 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.353140    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.354453 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.354434    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zq9td\""
Sep 18 15:34:39.354554 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.354482    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 15:34:39.354768 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.354752    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 15:34:39.355116 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.355077    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 15:34:39.355814 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.355384    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 15:34:39.355814 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.355561    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 15:34:39.355814 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.355731    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 15:34:39.356284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.356026    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dmvm6\""
Sep 18 15:34:39.356284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.356071    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 15:34:39.363859 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.362980    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 15:34:39.363859 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.363400    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 15:34:39.363859 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.363457    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 15:34:39.363859 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.363706    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 15:34:39.364879 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.364286    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tdnv7\""
Sep 18 15:34:39.364879 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.364479    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 15:34:39.364879 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.364559    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 15:34:39.367983 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.367822    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 15:34:39.435647 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.435454    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gqzcg"
Sep 18 15:34:39.441857 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.441780    2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 15:34:39.448631 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.448609    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gqzcg"
Sep 18 15:34:39.450295 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450270    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-run-multus-certs\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.450362 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450309    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fbfq\" (UniqueName: \"kubernetes.io/projected/846e19da-3e22-44c5-a6d5-5dd121781a03-kube-api-access-8fbfq\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.450362 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450334    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-registration-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.450433 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450366    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-run-ovn\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.450433 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450390    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a474076-cb9a-4eb3-8f05-c29502e12b38-env-overrides\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.450433 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450413    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9eb2cbd7-9c45-4158-88f1-39cc90d66fc4-host\") pod \"node-ca-hhm7w\" (UID: \"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4\") " pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.450518 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450436    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:39.450518 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450461    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-var-lib-kubelet\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.450518 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450487    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6a044247-0dee-476f-b2e4-309e4e0982d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.450518 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450514    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-system-cni-dir\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.450635 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450537    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-var-lib-cni-bin\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.450635 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450559    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-etc-selinux\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.450635 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450591    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-sys-fs\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.450635 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450615    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee563ec3-1031-4996-8ccb-5a3e8a973a10-host-slash\") pod \"iptables-alerter-4lpkq\" (UID: \"ee563ec3-1031-4996-8ccb-5a3e8a973a10\") " pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.450744 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450644    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgnms\" (UniqueName: \"kubernetes.io/projected/edf901bb-90cc-4841-88db-6de9d72eea52-kube-api-access-kgnms\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:39.450744 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450653    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-run-multus-certs\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.450744 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450672    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-cni-dir\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.450744 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450693    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-run\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.450744 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450718    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a044247-0dee-476f-b2e4-309e4e0982d8-system-cni-dir\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.450744 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450740    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpkzr\" (UniqueName: \"kubernetes.io/projected/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-kube-api-access-vpkzr\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450765    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f3967c88-e022-44a2-919c-468d1f757435-kubelet-config\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450789    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450812    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1-hosts-file\") pod \"node-resolver-rnzfc\" (UID: \"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1\") " pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450852    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6a044247-0dee-476f-b2e4-309e4e0982d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450878    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450902    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450935    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a474076-cb9a-4eb3-8f05-c29502e12b38-ovnkube-config\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450958    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-kubernetes\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450963    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-registration-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.450986    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-slash\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.451025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451010    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x6xw\" (UniqueName: \"kubernetes.io/projected/5a474076-cb9a-4eb3-8f05-c29502e12b38-kube-api-access-8x6xw\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451040    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-var-lib-kubelet\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451041    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-var-lib-cni-multus\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451084    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-etc-kubernetes\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451090    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-var-lib-cni-multus\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451108    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a044247-0dee-476f-b2e4-309e4e0982d8-cnibin\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451134    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkhcr\" (UniqueName: \"kubernetes.io/projected/6a044247-0dee-476f-b2e4-309e4e0982d8-kube-api-access-zkhcr\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451142    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a044247-0dee-476f-b2e4-309e4e0982d8-system-cni-dir\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451158    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae988a2f-8f5c-4506-a5ba-ba8d7d79212e-agent-certs\") pod \"konnectivity-agent-dl59b\" (UID: \"ae988a2f-8f5c-4506-a5ba-ba8d7d79212e\") " pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451191    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-kubelet\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451215    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-systemd-units\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451242    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee563ec3-1031-4996-8ccb-5a3e8a973a10-iptables-alerter-script\") pod \"iptables-alerter-4lpkq\" (UID: \"ee563ec3-1031-4996-8ccb-5a3e8a973a10\") " pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451264    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-sysctl-d\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451288    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djx58\" (UniqueName: \"kubernetes.io/projected/b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1-kube-api-access-djx58\") pod \"node-resolver-rnzfc\" (UID: \"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1\") " pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451321    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-socket-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451344    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-var-lib-openvswitch\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451367    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-node-log\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.451562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451392    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-run-netns\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451429    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9eb2cbd7-9c45-4158-88f1-39cc90d66fc4-serviceca\") pod \"node-ca-hhm7w\" (UID: \"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4\") " pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451451    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f3967c88-e022-44a2-919c-468d1f757435-dbus\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451456    2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451476    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1-tmp-dir\") pod \"node-resolver-rnzfc\" (UID: \"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1\") " pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451511    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-device-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451535    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-run-netns\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451574    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/846e19da-3e22-44c5-a6d5-5dd121781a03-cni-binary-copy\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451598    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-run-openvswitch\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451622    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24v5f\" (UniqueName: \"kubernetes.io/projected/9eb2cbd7-9c45-4158-88f1-39cc90d66fc4-kube-api-access-24v5f\") pod \"node-ca-hhm7w\" (UID: \"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4\") " pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451643    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-host\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451666    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-cnibin\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn2wb\" (UniqueName: \"kubernetes.io/projected/070e3db3-8130-4977-843e-9d520bb22197-kube-api-access-rn2wb\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451718    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-etc-openvswitch\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451743    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-run-ovn-kubernetes\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451770    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-sysctl-conf\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451796    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a044247-0dee-476f-b2e4-309e4e0982d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.452358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451960    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a044247-0dee-476f-b2e4-309e4e0982d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.451994    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a474076-cb9a-4eb3-8f05-c29502e12b38-ovn-node-metrics-cert\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452020    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-run-systemd\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452044    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-cni-netd\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452069    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a474076-cb9a-4eb3-8f05-c29502e12b38-ovnkube-script-lib\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452097    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-socket-dir-parent\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452126    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-run-k8s-cni-cncf-io\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452151    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-sysconfig\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452173    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-systemd\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452197    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-var-lib-kubelet\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452223    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a044247-0dee-476f-b2e4-309e4e0982d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452258    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae988a2f-8f5c-4506-a5ba-ba8d7d79212e-konnectivity-ca\") pod \"konnectivity-agent-dl59b\" (UID: \"ae988a2f-8f5c-4506-a5ba-ba8d7d79212e\") " pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452305    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-hostroot\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452330    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-daemon-config\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452336    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6a044247-0dee-476f-b2e4-309e4e0982d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452359    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl464\" (UniqueName: \"kubernetes.io/projected/ee563ec3-1031-4996-8ccb-5a3e8a973a10-kube-api-access-xl464\") pod \"iptables-alerter-4lpkq\" (UID: \"ee563ec3-1031-4996-8ccb-5a3e8a973a10\") " pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452385    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-sys\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.453164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452409    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-lib-modules\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452423    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-system-cni-dir\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452433    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-tuned\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452460    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-conf-dir\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452470    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-var-lib-cni-bin\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452489    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-log-socket\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452515    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-cni-bin\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452540    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452566    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-etc-selinux\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452569    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-modprobe-d\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452625    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-sys-fs\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452652    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-tmp\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452742    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-cnibin\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452790    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452855    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-run-netns\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452932    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-device-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.452977    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/070e3db3-8130-4977-843e-9d520bb22197-socket-dir\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.453980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453065    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6a044247-0dee-476f-b2e4-309e4e0982d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453358    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-socket-dir-parent\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453412    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-host-run-k8s-cni-cncf-io\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453436    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/846e19da-3e22-44c5-a6d5-5dd121781a03-cni-binary-copy\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453513    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a044247-0dee-476f-b2e4-309e4e0982d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453632    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-hostroot\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453675    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-etc-kubernetes\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453786    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-cni-dir\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453830    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-conf-dir\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453880    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a044247-0dee-476f-b2e4-309e4e0982d8-cnibin\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453946    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae988a2f-8f5c-4506-a5ba-ba8d7d79212e-konnectivity-ca\") pod \"konnectivity-agent-dl59b\" (UID: \"ae988a2f-8f5c-4506-a5ba-ba8d7d79212e\") " pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:39.454807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.453992    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/846e19da-3e22-44c5-a6d5-5dd121781a03-multus-daemon-config\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.458337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.458314    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae988a2f-8f5c-4506-a5ba-ba8d7d79212e-agent-certs\") pod \"konnectivity-agent-dl59b\" (UID: \"ae988a2f-8f5c-4506-a5ba-ba8d7d79212e\") " pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:39.472055 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.471989    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn2wb\" (UniqueName: \"kubernetes.io/projected/070e3db3-8130-4977-843e-9d520bb22197-kube-api-access-rn2wb\") pod \"aws-ebs-csi-driver-node-wxgzb\" (UID: \"070e3db3-8130-4977-843e-9d520bb22197\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.472886 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.472832    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fbfq\" (UniqueName: \"kubernetes.io/projected/846e19da-3e22-44c5-a6d5-5dd121781a03-kube-api-access-8fbfq\") pod \"multus-bcszs\" (UID: \"846e19da-3e22-44c5-a6d5-5dd121781a03\") " pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.476262 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.476223    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkhcr\" (UniqueName: \"kubernetes.io/projected/6a044247-0dee-476f-b2e4-309e4e0982d8-kube-api-access-zkhcr\") pod \"multus-additional-cni-plugins-ddhnl\" (UID: \"6a044247-0dee-476f-b2e4-309e4e0982d8\") " pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.522451 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.522399    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal" event={"ID":"f7aedb2ec9046cdbc367d9c6b6d51d18","Type":"ContainerStarted","Data":"6abfbea62de89ec71f7da450340a50d256656edb661976bb242d66220e00b68e"}
Sep 18 15:34:39.524220 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.524199    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal" event={"ID":"aa0a25f1f0b432907b52509419f94e73","Type":"ContainerStarted","Data":"9c7000f1fabf12c2b7d926514cde886a3dfd53dd3ecff93268f5f6d2bfbd7dd0"}
Sep 18 15:34:39.553696 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.553672    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpkzr\" (UniqueName: \"kubernetes.io/projected/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-kube-api-access-vpkzr\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.553831 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.553703    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f3967c88-e022-44a2-919c-468d1f757435-kubelet-config\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:39.553831 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.553728    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:39.553831 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.553801    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f3967c88-e022-44a2-919c-468d1f757435-kubelet-config\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:39.554053 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.553834    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:39.554053 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.553922    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret podName:f3967c88-e022-44a2-919c-468d1f757435 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:40.053898761 +0000 UTC m=+4.712691401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret") pod "global-pull-secret-syncer-bvgpr" (UID: "f3967c88-e022-44a2-919c-468d1f757435") : object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:39.554053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554012    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1-hosts-file\") pod \"node-resolver-rnzfc\" (UID: \"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1\") " pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.554053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554044    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554063    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a474076-cb9a-4eb3-8f05-c29502e12b38-ovnkube-config\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554077    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-kubernetes\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554104    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-slash\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554125    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x6xw\" (UniqueName: \"kubernetes.io/projected/5a474076-cb9a-4eb3-8f05-c29502e12b38-kube-api-access-8x6xw\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554129    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1-hosts-file\") pod \"node-resolver-rnzfc\" (UID: \"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1\") " pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554143    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-kubelet\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554174    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-kubelet\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554179    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-systemd-units\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554207    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee563ec3-1031-4996-8ccb-5a3e8a973a10-iptables-alerter-script\") pod \"iptables-alerter-4lpkq\" (UID: \"ee563ec3-1031-4996-8ccb-5a3e8a973a10\") " pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554232    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-sysctl-d\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.554253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554255    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djx58\" (UniqueName: \"kubernetes.io/projected/b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1-kube-api-access-djx58\") pod \"node-resolver-rnzfc\" (UID: \"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1\") " pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554281    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-var-lib-openvswitch\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554306    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-node-log\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554328    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9eb2cbd7-9c45-4158-88f1-39cc90d66fc4-serviceca\") pod \"node-ca-hhm7w\" (UID: \"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4\") " pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554351    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f3967c88-e022-44a2-919c-468d1f757435-dbus\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554374    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1-tmp-dir\") pod \"node-resolver-rnzfc\" (UID: \"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1\") " pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554399    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-run-netns\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554424    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-run-openvswitch\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554447    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24v5f\" (UniqueName: \"kubernetes.io/projected/9eb2cbd7-9c45-4158-88f1-39cc90d66fc4-kube-api-access-24v5f\") pod \"node-ca-hhm7w\" (UID: \"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4\") " pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554471    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-host\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554500    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-etc-openvswitch\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554525    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-run-ovn-kubernetes\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554548    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-sysctl-conf\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554573    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a474076-cb9a-4eb3-8f05-c29502e12b38-ovn-node-metrics-cert\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554597    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-run-systemd\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554622    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-cni-netd\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554647    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a474076-cb9a-4eb3-8f05-c29502e12b38-ovnkube-script-lib\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.554802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554672    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-sysconfig\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554695    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-systemd\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554717    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-var-lib-kubelet\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554748    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl464\" (UniqueName: \"kubernetes.io/projected/ee563ec3-1031-4996-8ccb-5a3e8a973a10-kube-api-access-xl464\") pod \"iptables-alerter-4lpkq\" (UID: \"ee563ec3-1031-4996-8ccb-5a3e8a973a10\") " pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554766    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-kubernetes\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554770    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-sys\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554810    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-lib-modules\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554816    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-sys\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554834    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-tuned\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554873    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-systemd-units\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554882    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-log-socket\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554911    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-cni-bin\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554936    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554961    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-modprobe-d\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554983    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-tmp\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555013    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-run-ovn\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555037    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a474076-cb9a-4eb3-8f05-c29502e12b38-env-overrides\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555060    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9eb2cbd7-9c45-4158-88f1-39cc90d66fc4-host\") pod \"node-ca-hhm7w\" (UID: \"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4\") " pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.555591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555083    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555117    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee563ec3-1031-4996-8ccb-5a3e8a973a10-host-slash\") pod \"iptables-alerter-4lpkq\" (UID: \"ee563ec3-1031-4996-8ccb-5a3e8a973a10\") " pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555142    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgnms\" (UniqueName: \"kubernetes.io/projected/edf901bb-90cc-4841-88db-6de9d72eea52-kube-api-access-kgnms\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555171    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-run\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555239    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-run\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555282    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-slash\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555389    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee563ec3-1031-4996-8ccb-5a3e8a973a10-iptables-alerter-script\") pod \"iptables-alerter-4lpkq\" (UID: \"ee563ec3-1031-4996-8ccb-5a3e8a973a10\") " pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554208    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555544    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-sysctl-d\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555569    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-lib-modules\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555776    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-var-lib-openvswitch\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.555827    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-node-log\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556031    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-sysctl-conf\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556091    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f3967c88-e022-44a2-919c-468d1f757435-dbus\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556233    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9eb2cbd7-9c45-4158-88f1-39cc90d66fc4-serviceca\") pod \"node-ca-hhm7w\" (UID: \"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4\") " pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556313    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1-tmp-dir\") pod \"node-resolver-rnzfc\" (UID: \"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1\") " pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556376    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-run-netns\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.556430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556421    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-run-openvswitch\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556617    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-host\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.557096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556661    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-etc-openvswitch\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556701    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-run-ovn-kubernetes\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556738    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-run-ovn\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556785    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9eb2cbd7-9c45-4158-88f1-39cc90d66fc4-host\") pod \"node-ca-hhm7w\" (UID: \"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4\") " pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.557096 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.556885    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:39.557096 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.556931    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs podName:edf901bb-90cc-4841-88db-6de9d72eea52 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:40.05691588 +0000 UTC m=+4.715708523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs") pod "network-metrics-daemon-4rnz2" (UID: "edf901bb-90cc-4841-88db-6de9d72eea52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:39.557096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.556971    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee563ec3-1031-4996-8ccb-5a3e8a973a10-host-slash\") pod \"iptables-alerter-4lpkq\" (UID: \"ee563ec3-1031-4996-8ccb-5a3e8a973a10\") " pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.557384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557108    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a474076-cb9a-4eb3-8f05-c29502e12b38-env-overrides\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557164    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-log-socket\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557189    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-sysconfig\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.557384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557207    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-cni-bin\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557248    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-run-systemd\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557286    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a474076-cb9a-4eb3-8f05-c29502e12b38-host-cni-netd\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557631 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557551    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-modprobe-d\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.557700 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557679    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a474076-cb9a-4eb3-8f05-c29502e12b38-ovnkube-script-lib\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557785 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557757    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-var-lib-kubelet\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.557853 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.554716    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a474076-cb9a-4eb3-8f05-c29502e12b38-ovnkube-config\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.557853 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.557814    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-systemd\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.558822 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.558382    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-etc-tuned\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.559488 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.559447    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a474076-cb9a-4eb3-8f05-c29502e12b38-ovn-node-metrics-cert\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.559746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.559723    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-tmp\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.572383 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.572331    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x6xw\" (UniqueName: \"kubernetes.io/projected/5a474076-cb9a-4eb3-8f05-c29502e12b38-kube-api-access-8x6xw\") pod \"ovnkube-node-6wqf6\" (UID: \"5a474076-cb9a-4eb3-8f05-c29502e12b38\") " pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.574463 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.574443    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djx58\" (UniqueName: \"kubernetes.io/projected/b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1-kube-api-access-djx58\") pod \"node-resolver-rnzfc\" (UID: \"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1\") " pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.577817 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.577795    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 15:34:39.577918 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.577823    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 15:34:39.577918 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.577849    2585 projected.go:196] Error preparing data for projected volume kube-api-access-86zbx for pod openshift-network-diagnostics/network-check-target-qq2zh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:39.577918 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:39.577913    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx podName:f64e3326-8578-4fc8-b8f4-4bf7689a4a61 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:40.077890078 +0000 UTC m=+4.736682721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-86zbx" (UniqueName: "kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx") pod "network-check-target-qq2zh" (UID: "f64e3326-8578-4fc8-b8f4-4bf7689a4a61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:39.581777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.580492    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl464\" (UniqueName: \"kubernetes.io/projected/ee563ec3-1031-4996-8ccb-5a3e8a973a10-kube-api-access-xl464\") pod \"iptables-alerter-4lpkq\" (UID: \"ee563ec3-1031-4996-8ccb-5a3e8a973a10\") " pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.581871 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.581789    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24v5f\" (UniqueName: \"kubernetes.io/projected/9eb2cbd7-9c45-4158-88f1-39cc90d66fc4-kube-api-access-24v5f\") pod \"node-ca-hhm7w\" (UID: \"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4\") " pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.582955 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.582901    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgnms\" (UniqueName: \"kubernetes.io/projected/edf901bb-90cc-4841-88db-6de9d72eea52-kube-api-access-kgnms\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:39.587268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.587240    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpkzr\" (UniqueName: \"kubernetes.io/projected/6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4-kube-api-access-vpkzr\") pod \"tuned-7hmc7\" (UID: \"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4\") " pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.652795 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.652755    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb"
Sep 18 15:34:39.662158 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.661991    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ddhnl"
Sep 18 15:34:39.663639 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:39.663613    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod070e3db3_8130_4977_843e_9d520bb22197.slice/crio-9b0ab6a0ccb2dc16611c0ca9d184040aa58ab6ad7b6080236272ee3dcd3f9eb1 WatchSource:0}: Error finding container 9b0ab6a0ccb2dc16611c0ca9d184040aa58ab6ad7b6080236272ee3dcd3f9eb1: Status 404 returned error can't find the container with id 9b0ab6a0ccb2dc16611c0ca9d184040aa58ab6ad7b6080236272ee3dcd3f9eb1
Sep 18 15:34:39.670929 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:39.670902    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a044247_0dee_476f_b2e4_309e4e0982d8.slice/crio-44efef82e4d0a6a361e1557ffd83123868e40a3bc2533f7d8384bf32461acc56 WatchSource:0}: Error finding container 44efef82e4d0a6a361e1557ffd83123868e40a3bc2533f7d8384bf32461acc56: Status 404 returned error can't find the container with id 44efef82e4d0a6a361e1557ffd83123868e40a3bc2533f7d8384bf32461acc56
Sep 18 15:34:39.674562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.674539    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:39.682825 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.682674    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bcszs"
Sep 18 15:34:39.683268 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:39.683243    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae988a2f_8f5c_4506_a5ba_ba8d7d79212e.slice/crio-ebbc5c10757ed412bf15707706f0da2222ab22e9e00d55dac4ef5af4281d548f WatchSource:0}: Error finding container ebbc5c10757ed412bf15707706f0da2222ab22e9e00d55dac4ef5af4281d548f: Status 404 returned error can't find the container with id ebbc5c10757ed412bf15707706f0da2222ab22e9e00d55dac4ef5af4281d548f
Sep 18 15:34:39.684585 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.684512    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 15:34:39.690855 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.689832    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7hmc7"
Sep 18 15:34:39.691252 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:39.691228    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod846e19da_3e22_44c5_a6d5_5dd121781a03.slice/crio-7f1534fed3cc3029d028f3e490716d9a700f6701bf7c222ab5637c2dfb09fd54 WatchSource:0}: Error finding container 7f1534fed3cc3029d028f3e490716d9a700f6701bf7c222ab5637c2dfb09fd54: Status 404 returned error can't find the container with id 7f1534fed3cc3029d028f3e490716d9a700f6701bf7c222ab5637c2dfb09fd54
Sep 18 15:34:39.698212 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.698168    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hhm7w"
Sep 18 15:34:39.699438 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:39.699413    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa0be55_1bc2_43e1_8fb7_04dcc3f833a4.slice/crio-2bf336c4a21c4c14f6e271d9d4ff5dc69c65b5290e3548a7bb820ebc05e03ba0 WatchSource:0}: Error finding container 2bf336c4a21c4c14f6e271d9d4ff5dc69c65b5290e3548a7bb820ebc05e03ba0: Status 404 returned error can't find the container with id 2bf336c4a21c4c14f6e271d9d4ff5dc69c65b5290e3548a7bb820ebc05e03ba0
Sep 18 15:34:39.706577 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.706561    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4lpkq"
Sep 18 15:34:39.706917 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:39.706895    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb2cbd7_9c45_4158_88f1_39cc90d66fc4.slice/crio-bcb784a9da511c403bcc8005b58863317c346c8a4ac98f44ca983748c032ccbf WatchSource:0}: Error finding container bcb784a9da511c403bcc8005b58863317c346c8a4ac98f44ca983748c032ccbf: Status 404 returned error can't find the container with id bcb784a9da511c403bcc8005b58863317c346c8a4ac98f44ca983748c032ccbf
Sep 18 15:34:39.714450 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:39.714428    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee563ec3_1031_4996_8ccb_5a3e8a973a10.slice/crio-8c23248190f841f35119102bc2191009015b1fd2837f6faed1db65e1c2b6b2bc WatchSource:0}: Error finding container 8c23248190f841f35119102bc2191009015b1fd2837f6faed1db65e1c2b6b2bc: Status 404 returned error can't find the container with id 8c23248190f841f35119102bc2191009015b1fd2837f6faed1db65e1c2b6b2bc
Sep 18 15:34:39.715581 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.715555    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rnzfc"
Sep 18 15:34:39.722321 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:39.722300    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:34:39.724915 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:39.724890    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73dcb9d_63e4_4ca8_9d26_9c3674c3e6d1.slice/crio-908dc513926baa665c3e22afb95de6acedfa77abefe7594462abe77b267987ee WatchSource:0}: Error finding container 908dc513926baa665c3e22afb95de6acedfa77abefe7594462abe77b267987ee: Status 404 returned error can't find the container with id 908dc513926baa665c3e22afb95de6acedfa77abefe7594462abe77b267987ee
Sep 18 15:34:40.058334 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.058256    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:40.058334 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.058291    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:40.058547 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:40.058393    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:40.058547 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:40.058399    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:40.058547 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:40.058443    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret podName:f3967c88-e022-44a2-919c-468d1f757435 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:41.058430915 +0000 UTC m=+5.717223557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret") pod "global-pull-secret-syncer-bvgpr" (UID: "f3967c88-e022-44a2-919c-468d1f757435") : object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:40.058547 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:40.058456    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs podName:edf901bb-90cc-4841-88db-6de9d72eea52 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:41.058450464 +0000 UTC m=+5.717243102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs") pod "network-metrics-daemon-4rnz2" (UID: "edf901bb-90cc-4841-88db-6de9d72eea52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:40.159195 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.159158    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:40.159374 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:40.159337    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 15:34:40.159374 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:40.159365    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 15:34:40.159477 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:40.159376    2585 projected.go:196] Error preparing data for projected volume kube-api-access-86zbx for pod openshift-network-diagnostics/network-check-target-qq2zh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:40.159477 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:40.159448    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx podName:f64e3326-8578-4fc8-b8f4-4bf7689a4a61 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:41.159427155 +0000 UTC m=+5.818219795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-86zbx" (UniqueName: "kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx") pod "network-check-target-qq2zh" (UID: "f64e3326-8578-4fc8-b8f4-4bf7689a4a61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:40.451261 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.451180    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 15:29:39 +0000 UTC" deadline="2028-05-11 18:27:31.235217884 +0000 UTC"
Sep 18 15:34:40.451261 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.451214    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14426h52m50.784008659s"
Sep 18 15:34:40.516893 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.516860    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:40.517051 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:40.516989    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:40.527716 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.527686    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4lpkq" event={"ID":"ee563ec3-1031-4996-8ccb-5a3e8a973a10","Type":"ContainerStarted","Data":"8c23248190f841f35119102bc2191009015b1fd2837f6faed1db65e1c2b6b2bc"}
Sep 18 15:34:40.529112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.529081    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hhm7w" event={"ID":"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4","Type":"ContainerStarted","Data":"bcb784a9da511c403bcc8005b58863317c346c8a4ac98f44ca983748c032ccbf"}
Sep 18 15:34:40.530516 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.530489    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7hmc7" event={"ID":"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4","Type":"ContainerStarted","Data":"2bf336c4a21c4c14f6e271d9d4ff5dc69c65b5290e3548a7bb820ebc05e03ba0"}
Sep 18 15:34:40.531893 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.531867    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dl59b" event={"ID":"ae988a2f-8f5c-4506-a5ba-ba8d7d79212e","Type":"ContainerStarted","Data":"ebbc5c10757ed412bf15707706f0da2222ab22e9e00d55dac4ef5af4281d548f"}
Sep 18 15:34:40.533396 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.533363    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb" event={"ID":"070e3db3-8130-4977-843e-9d520bb22197","Type":"ContainerStarted","Data":"9b0ab6a0ccb2dc16611c0ca9d184040aa58ab6ad7b6080236272ee3dcd3f9eb1"}
Sep 18 15:34:40.534643 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.534622    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bcszs" event={"ID":"846e19da-3e22-44c5-a6d5-5dd121781a03","Type":"ContainerStarted","Data":"7f1534fed3cc3029d028f3e490716d9a700f6701bf7c222ab5637c2dfb09fd54"}
Sep 18 15:34:40.535907 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.535861    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" event={"ID":"6a044247-0dee-476f-b2e4-309e4e0982d8","Type":"ContainerStarted","Data":"44efef82e4d0a6a361e1557ffd83123868e40a3bc2533f7d8384bf32461acc56"}
Sep 18 15:34:40.537124 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:40.537104    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rnzfc" event={"ID":"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1","Type":"ContainerStarted","Data":"908dc513926baa665c3e22afb95de6acedfa77abefe7594462abe77b267987ee"}
Sep 18 15:34:40.757632 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:34:40.757604    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a474076_cb9a_4eb3_8f05_c29502e12b38.slice/crio-a77eb207c2d7b8c9f5be5716746faf284da98636005aa41206fd0a2030633482 WatchSource:0}: Error finding container a77eb207c2d7b8c9f5be5716746faf284da98636005aa41206fd0a2030633482: Status 404 returned error can't find the container with id a77eb207c2d7b8c9f5be5716746faf284da98636005aa41206fd0a2030633482
Sep 18 15:34:41.068081 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.068001    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:41.068081 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.068057    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:41.068302 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.068184    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:41.068302 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.068244    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret podName:f3967c88-e022-44a2-919c-468d1f757435 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:43.068226525 +0000 UTC m=+7.727019168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret") pod "global-pull-secret-syncer-bvgpr" (UID: "f3967c88-e022-44a2-919c-468d1f757435") : object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:41.068680 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.068660    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:41.068744 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.068716    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs podName:edf901bb-90cc-4841-88db-6de9d72eea52 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:43.068701356 +0000 UTC m=+7.727493999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs") pod "network-metrics-daemon-4rnz2" (UID: "edf901bb-90cc-4841-88db-6de9d72eea52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:41.169523 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.169482    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:41.169702 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.169679    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 15:34:41.169702 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.169696    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 15:34:41.169818 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.169708    2585 projected.go:196] Error preparing data for projected volume kube-api-access-86zbx for pod openshift-network-diagnostics/network-check-target-qq2zh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:41.169818 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.169766    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx podName:f64e3326-8578-4fc8-b8f4-4bf7689a4a61 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:43.169748665 +0000 UTC m=+7.828541303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-86zbx" (UniqueName: "kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx") pod "network-check-target-qq2zh" (UID: "f64e3326-8578-4fc8-b8f4-4bf7689a4a61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:41.452373 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.452257    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 15:29:39 +0000 UTC" deadline="2028-04-11 11:50:05.391271569 +0000 UTC"
Sep 18 15:34:41.452373 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.452296    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13700h15m23.938979651s"
Sep 18 15:34:41.519635 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.517958    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:41.519635 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.518105    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:41.519635 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.518556    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:41.519635 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:41.518644    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:41.550863 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.550662    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal" event={"ID":"aa0a25f1f0b432907b52509419f94e73","Type":"ContainerStarted","Data":"8ce58e557e94c8df021689169f02db4c8e781e6d89ff3657ff2231825acb8fce"}
Sep 18 15:34:41.552684 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.552662    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerStarted","Data":"a77eb207c2d7b8c9f5be5716746faf284da98636005aa41206fd0a2030633482"}
Sep 18 15:34:41.573530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:41.573481    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-208.ec2.internal" podStartSLOduration=3.573465261 podStartE2EDuration="3.573465261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:34:41.572366492 +0000 UTC m=+6.231159153" watchObservedRunningTime="2026-09-18 15:34:41.573465261 +0000 UTC m=+6.232257921"
Sep 18 15:34:42.517824 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:42.517498    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:42.518276 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:42.517867    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:42.570904 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:42.570809    2585 generic.go:356] "Generic (PLEG): container finished" podID="f7aedb2ec9046cdbc367d9c6b6d51d18" containerID="10a58f74886800cee40b0293345f87b358470a78fc84be3793ebb11464ae0a19" exitCode=0
Sep 18 15:34:42.571068 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:42.570916    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal" event={"ID":"f7aedb2ec9046cdbc367d9c6b6d51d18","Type":"ContainerDied","Data":"10a58f74886800cee40b0293345f87b358470a78fc84be3793ebb11464ae0a19"}
Sep 18 15:34:42.584597 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:42.584575    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dl59b" event={"ID":"ae988a2f-8f5c-4506-a5ba-ba8d7d79212e","Type":"ContainerStarted","Data":"ebbac232af157ffb202fd5fd40589a6e3043e6b38a9eea142ad9af9f914f07ab"}
Sep 18 15:34:43.089296 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:43.089222    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:43.089296 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:43.089278    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:43.089559 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.089430    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:43.089559 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.089485    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret podName:f3967c88-e022-44a2-919c-468d1f757435 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:47.089467878 +0000 UTC m=+11.748260523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret") pod "global-pull-secret-syncer-bvgpr" (UID: "f3967c88-e022-44a2-919c-468d1f757435") : object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:43.089930 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.089910    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:43.090020 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.089969    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs podName:edf901bb-90cc-4841-88db-6de9d72eea52 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:47.089953161 +0000 UTC m=+11.748745805 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs") pod "network-metrics-daemon-4rnz2" (UID: "edf901bb-90cc-4841-88db-6de9d72eea52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:43.190295 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:43.190247    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:43.190471 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.190446    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 15:34:43.190471 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.190465    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 15:34:43.190585 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.190477    2585 projected.go:196] Error preparing data for projected volume kube-api-access-86zbx for pod openshift-network-diagnostics/network-check-target-qq2zh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:43.190585 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.190528    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx podName:f64e3326-8578-4fc8-b8f4-4bf7689a4a61 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:47.190511277 +0000 UTC m=+11.849303920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-86zbx" (UniqueName: "kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx") pod "network-check-target-qq2zh" (UID: "f64e3326-8578-4fc8-b8f4-4bf7689a4a61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:43.517793 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:43.517708    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:43.517960 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:43.517708    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:43.517960 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.517860    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:43.517960 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:43.517906    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:43.590505 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:43.590470    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal" event={"ID":"f7aedb2ec9046cdbc367d9c6b6d51d18","Type":"ContainerStarted","Data":"1772975c4cb7ffa4c0fb0e55fe89a5f65d0f288861f652796b9c8b336053d4d6"}
Sep 18 15:34:43.618905 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:43.618832    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dl59b" podStartSLOduration=4.489328931 podStartE2EDuration="6.618810901s" totalImagesPullingTime="2.12948197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:34:42.626321929 +0000 UTC m=+7.285114588" watchObservedRunningTime="2026-09-18 15:34:43.618810901 +0000 UTC m=+8.277603562"
Sep 18 15:34:44.517564 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:44.517526    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:44.517730 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:44.517654    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:45.518433 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:45.517497    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:45.518433 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:45.517536    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:45.518433 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:45.517641    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:45.518433 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:45.517785    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:46.517332 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:46.517261    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:46.517572 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:46.517391    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:46.789530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:46.789462    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:46.790628 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:46.790584    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:46.828296 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:46.828245    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-208.ec2.internal" podStartSLOduration=8.828230323 podStartE2EDuration="8.828230323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:34:43.618757238 +0000 UTC m=+8.277549898" watchObservedRunningTime="2026-09-18 15:34:46.828230323 +0000 UTC m=+11.487022983"
Sep 18 15:34:47.121716 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:47.121637    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:47.121716 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:47.121693    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:47.121944 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.121871    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:47.121944 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.121938    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret podName:f3967c88-e022-44a2-919c-468d1f757435 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:55.121919096 +0000 UTC m=+19.780711746 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret") pod "global-pull-secret-syncer-bvgpr" (UID: "f3967c88-e022-44a2-919c-468d1f757435") : object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:47.122444 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.122327    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:47.122444 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.122410    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs podName:edf901bb-90cc-4841-88db-6de9d72eea52 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:55.122374694 +0000 UTC m=+19.781167335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs") pod "network-metrics-daemon-4rnz2" (UID: "edf901bb-90cc-4841-88db-6de9d72eea52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:47.222827 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:47.222786    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:47.223012 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.222974    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 15:34:47.223012 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.223000    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 15:34:47.223012 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.223013    2585 projected.go:196] Error preparing data for projected volume kube-api-access-86zbx for pod openshift-network-diagnostics/network-check-target-qq2zh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:47.223133 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.223079    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx podName:f64e3326-8578-4fc8-b8f4-4bf7689a4a61 nodeName:}" failed. No retries permitted until 2026-09-18 15:34:55.223059583 +0000 UTC m=+19.881852246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-86zbx" (UniqueName: "kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx") pod "network-check-target-qq2zh" (UID: "f64e3326-8578-4fc8-b8f4-4bf7689a4a61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:47.518161 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:47.518087    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:47.518317 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.518199    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:47.518457 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:47.518433    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:47.518590 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:47.518559    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:47.599901 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:47.599866    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:47.600555 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:47.600538    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dl59b"
Sep 18 15:34:48.516691 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:48.516656    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:48.517133 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:48.516795    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:49.517814 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:49.517505    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:49.517814 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:49.517636    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:49.517814 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:49.517655    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:49.517814 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:49.517777    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:50.517412 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:50.517369    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:50.517651 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:50.517492    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:51.517108 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:51.517070    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:51.517108 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:51.517088    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:51.517640 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:51.517225    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:51.517640 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:51.517362    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:52.000040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:51.999950    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:52.182673 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:52.182639    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:52.517638 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:52.517603    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:52.518028 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:52.517726    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:53.516951 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:53.516905    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:53.517125 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:53.517038    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:53.517125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:53.516904    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:53.517263 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:53.517144    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:54.517091 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:54.517064    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:54.517466 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:54.517178    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:55.184211 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:55.184171    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:55.184211 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:55.184214    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:55.184462 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.184320    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:55.184462 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.184333    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:55.184462 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.184382    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret podName:f3967c88-e022-44a2-919c-468d1f757435 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:11.18436359 +0000 UTC m=+35.843156245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret") pod "global-pull-secret-syncer-bvgpr" (UID: "f3967c88-e022-44a2-919c-468d1f757435") : object "kube-system"/"original-pull-secret" not registered
Sep 18 15:34:55.184462 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.184401    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs podName:edf901bb-90cc-4841-88db-6de9d72eea52 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:11.184391458 +0000 UTC m=+35.843184096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs") pod "network-metrics-daemon-4rnz2" (UID: "edf901bb-90cc-4841-88db-6de9d72eea52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:34:55.284964 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:55.284932    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:55.285156 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.285056    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 15:34:55.285156 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.285070    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 15:34:55.285156 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.285079    2585 projected.go:196] Error preparing data for projected volume kube-api-access-86zbx for pod openshift-network-diagnostics/network-check-target-qq2zh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:55.285156 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.285133    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx podName:f64e3326-8578-4fc8-b8f4-4bf7689a4a61 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:11.285115064 +0000 UTC m=+35.943907719 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-86zbx" (UniqueName: "kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx") pod "network-check-target-qq2zh" (UID: "f64e3326-8578-4fc8-b8f4-4bf7689a4a61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:34:55.517326 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:55.517248    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:55.517745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:55.517246    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:55.517745 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.517377    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:55.517745 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:55.517468    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:56.517585 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:56.517556    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:56.518031 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:56.517681    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:56.766465 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:56.766434    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:56.874412 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:56.874328    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:57.020622 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:57.020595    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:57.518642 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:57.518611    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:57.519024 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:57.518728    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:57.519372 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:57.519222    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:57.519372 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:57.519336    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:57.620441 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:57.620409    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rnzfc" event={"ID":"b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1","Type":"ContainerStarted","Data":"0ef67b265d19e38141af2347274aeef92249c880c8847cf845198d900323e282"}
Sep 18 15:34:57.622243 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:57.622218    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hhm7w" event={"ID":"9eb2cbd7-9c45-4158-88f1-39cc90d66fc4","Type":"ContainerStarted","Data":"f2e3138b5ada7ef5b4d22ca7f3b0d24aef2b4df42f148443fc427a9bbb07b4d5"}
Sep 18 15:34:57.627512 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:57.627454    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb" event={"ID":"070e3db3-8130-4977-843e-9d520bb22197","Type":"ContainerStarted","Data":"160e6947795f4255846da8160055e6c6d2a833ae8ef737c055d87f0b03b067de"}
Sep 18 15:34:57.629468 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:57.629445    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" event={"ID":"6a044247-0dee-476f-b2e4-309e4e0982d8","Type":"ContainerStarted","Data":"905d70e0d0af1d566e80a05db8e474462fd7d435b897407e39332cb0f192af43"}
Sep 18 15:34:57.656004 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:57.655963    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rnzfc" podStartSLOduration=2.98884871 podStartE2EDuration="20.655949273s" totalImagesPullingTime="17.667100563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:34:57.65591625 +0000 UTC m=+22.314708910" watchObservedRunningTime="2026-09-18 15:34:57.655949273 +0000 UTC m=+22.314741932"
Sep 18 15:34:57.919990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:57.919822    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:58.095210 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.095181    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:58.517663 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.517466    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:34:58.517809 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:58.517755    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:34:58.633324 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.633290    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7hmc7" event={"ID":"6aa0be55-1bc2-43e1-8fb7-04dcc3f833a4","Type":"ContainerStarted","Data":"bb93d6774ba52d269bfe586a5899270bd69284fa4ef56787b5837bb139f75971"}
Sep 18 15:34:58.634786 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.634757    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bcszs" event={"ID":"846e19da-3e22-44c5-a6d5-5dd121781a03","Type":"ContainerStarted","Data":"99f1a1d31087c534f863ecffb3d4c7f679059ac71b53878a87b8dcd51f85ed20"}
Sep 18 15:34:58.636189 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.636168    2585 generic.go:356] "Generic (PLEG): container finished" podID="6a044247-0dee-476f-b2e4-309e4e0982d8" containerID="905d70e0d0af1d566e80a05db8e474462fd7d435b897407e39332cb0f192af43" exitCode=0
Sep 18 15:34:58.636291 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.636238    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" event={"ID":"6a044247-0dee-476f-b2e4-309e4e0982d8","Type":"ContainerDied","Data":"905d70e0d0af1d566e80a05db8e474462fd7d435b897407e39332cb0f192af43"}
Sep 18 15:34:58.639095 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.639074    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:34:58.639454 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.639432    2585 generic.go:356] "Generic (PLEG): container finished" podID="5a474076-cb9a-4eb3-8f05-c29502e12b38" containerID="c4fbaeb190614f7a2518f7fad2a2a8efd77302cf72631451d0e67cf7b8670860" exitCode=1
Sep 18 15:34:58.639522 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.639500    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerStarted","Data":"04b67f99d0fd5e530b959927eb96815340b39deb24463100bf448ba531d120ae"}
Sep 18 15:34:58.639571 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.639558    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerStarted","Data":"25df76f1f242fbf59a8adc6264ffc675e400829e87949c72db9aecfd778f0886"}
Sep 18 15:34:58.639605 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.639589    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerStarted","Data":"bfd1156513e51b411cd97c67c1e7c99cf6db617f01ba293f10a2468662c01117"}
Sep 18 15:34:58.639639 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.639616    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerStarted","Data":"0149b285161010abcd2c25fcafc34dbbafe05c6ed5852f1dc220ca72c706d718"}
Sep 18 15:34:58.639668 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.639643    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerDied","Data":"c4fbaeb190614f7a2518f7fad2a2a8efd77302cf72631451d0e67cf7b8670860"}
Sep 18 15:34:58.639697 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.639673    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerStarted","Data":"406180dfb0df1a556174adee26aee6ed4ffd4308056882af48113baaa6ab9e51"}
Sep 18 15:34:58.667436 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.667397    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hhm7w" podStartSLOduration=11.388299987 podStartE2EDuration="21.667385801s" totalImagesPullingTime="10.279085814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:34:57.727646171 +0000 UTC m=+22.386438842" watchObservedRunningTime="2026-09-18 15:34:58.667385801 +0000 UTC m=+23.326178461"
Sep 18 15:34:58.706943 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.706903    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bcszs" podStartSLOduration=3.968828519 podStartE2EDuration="21.706887135s" totalImagesPullingTime="17.738058616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:34:58.706569788 +0000 UTC m=+23.365362449" watchObservedRunningTime="2026-09-18 15:34:58.706887135 +0000 UTC m=+23.365679799"
Sep 18 15:34:58.707142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.707120    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7hmc7" podStartSLOduration=3.978368466 podStartE2EDuration="21.707110544s" totalImagesPullingTime="17.728742078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:34:58.667594886 +0000 UTC m=+23.326387545" watchObservedRunningTime="2026-09-18 15:34:58.707110544 +0000 UTC m=+23.365903238"
Sep 18 15:34:58.979366 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:58.979338    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:59.012768 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.012744    2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 15:34:59.104025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.103994    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:59.218923 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.218830    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:59.429458 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.429305    2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T15:34:59.012765872Z","UUID":"8655f959-3d95-4fb0-8b66-d18b69850942","Handler":null,"Name":"","Endpoint":""}
Sep 18 15:34:59.430931 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.430912    2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 15:34:59.431062 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.430943    2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 15:34:59.517576 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.517512    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:34:59.517726 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:59.517611    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:34:59.518046 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.518023    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:34:59.518152 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:34:59.518133    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:34:59.644904 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.644866    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb" event={"ID":"070e3db3-8130-4977-843e-9d520bb22197","Type":"ContainerStarted","Data":"bc4920acb994d97334d438b2316f15d899b088afbff1ef3cb105c5c7f4ce5794"}
Sep 18 15:34:59.647058 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.647022    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4lpkq" event={"ID":"ee563ec3-1031-4996-8ccb-5a3e8a973a10","Type":"ContainerStarted","Data":"a3535ea4a6693b0ee7ffd44d55b649a548cae8bc5ad80b4e20a870c73b856f69"}
Sep 18 15:34:59.804299 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.804221    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:34:59.892765 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:34:59.892739    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log" limit=1024
Sep 18 15:35:00.517412 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:00.517384    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:00.517503 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:00.517476    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:35:00.651441 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:00.651403    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb" event={"ID":"070e3db3-8130-4977-843e-9d520bb22197","Type":"ContainerStarted","Data":"3b1f8d16f175851686b96751e00744e24e6e7615aaa85431236690be39b04d80"}
Sep 18 15:35:00.654690 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:00.654669    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:35:00.655025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:00.655007    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerStarted","Data":"fe8180239a68ea74356b2179274bcd3ba5ab5bb61ac7deba26cfd9430a02511c"}
Sep 18 15:35:00.697790 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:00.697736    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4lpkq" podStartSLOduration=6.020423957 podStartE2EDuration="23.697717867s" totalImagesPullingTime="17.67729391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:34:59.666906089 +0000 UTC m=+24.325698751" watchObservedRunningTime="2026-09-18 15:35:00.697717867 +0000 UTC m=+25.356510528"
Sep 18 15:35:00.698114 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:00.698088    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wxgzb" podStartSLOduration=3.512206317 podStartE2EDuration="23.698079052s" totalImagesPullingTime="20.185872735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:00.697622844 +0000 UTC m=+25.356415504" watchObservedRunningTime="2026-09-18 15:35:00.698079052 +0000 UTC m=+25.356871716"
Sep 18 15:35:01.517631 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:01.517605    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:01.517788 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:01.517605    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:01.517788 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:01.517713    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:35:01.517788 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:01.517760    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:35:02.516746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:02.516585    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:02.517208 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:02.516846    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:35:03.517781 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:03.517750    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:03.518184 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:03.517911    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:35:03.518184 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:03.517958    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:03.518184 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:03.518070    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:35:03.665162 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:03.665138    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:35:03.665519 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:03.665496    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerStarted","Data":"c0d8d4b45860da4cc31714d09c73b94e14212655149b5f02a30f7c10f8236daa"}
Sep 18 15:35:03.665892 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:03.665750    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:35:03.665998 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:03.665981    2585 scope.go:117] "RemoveContainer" containerID="c4fbaeb190614f7a2518f7fad2a2a8efd77302cf72631451d0e67cf7b8670860"
Sep 18 15:35:03.686086 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:03.685993    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:35:04.517260 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:04.517238    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:04.517386 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:04.517330    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:35:04.669705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:04.669669    2585 generic.go:356] "Generic (PLEG): container finished" podID="6a044247-0dee-476f-b2e4-309e4e0982d8" containerID="65f248965ccd3344d020c3def8dbacd0e3b2b7df167590dfeca883fa5297ff78" exitCode=0
Sep 18 15:35:04.670353 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:04.669744    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" event={"ID":"6a044247-0dee-476f-b2e4-309e4e0982d8","Type":"ContainerDied","Data":"65f248965ccd3344d020c3def8dbacd0e3b2b7df167590dfeca883fa5297ff78"}
Sep 18 15:35:04.673528 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:04.673512    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:35:04.673903 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:04.673881    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" event={"ID":"5a474076-cb9a-4eb3-8f05-c29502e12b38","Type":"ContainerStarted","Data":"bc6f3bccd915f359ae605a44399c776255cb6ba0189820b19f16e54014cef58f"}
Sep 18 15:35:04.673997 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:04.673967    2585 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 18 15:35:04.674173 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:04.674157    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:35:04.688506 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:04.688485    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:35:04.759895 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:04.759778    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6" podStartSLOduration=10.703983647 podStartE2EDuration="27.759762346s" totalImagesPullingTime="17.055778699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:04.759616536 +0000 UTC m=+29.418409208" watchObservedRunningTime="2026-09-18 15:35:04.759762346 +0000 UTC m=+29.418555010"
Sep 18 15:35:05.517499 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.517346    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:05.517626 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.517412    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:05.517626 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:05.517581    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:35:05.517707 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:05.517650    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:35:05.677627 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.677529    2585 generic.go:356] "Generic (PLEG): container finished" podID="6a044247-0dee-476f-b2e4-309e4e0982d8" containerID="5dbd55962a0396a4fde6817d953035ba3f865bbf40ce5f20be2b073d9e90411c" exitCode=0
Sep 18 15:35:05.678000 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.677618    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" event={"ID":"6a044247-0dee-476f-b2e4-309e4e0982d8","Type":"ContainerDied","Data":"5dbd55962a0396a4fde6817d953035ba3f865bbf40ce5f20be2b073d9e90411c"}
Sep 18 15:35:05.678000 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.677788    2585 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 18 15:35:05.713599 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.713577    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qq2zh"]
Sep 18 15:35:05.713704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.713661    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:05.713746 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:05.713731    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:35:05.721942 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.721918    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:35:05.722576 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.722556    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4rnz2"]
Sep 18 15:35:05.722642 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.722632    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:05.722715 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:05.722702    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:35:05.745495 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.745477    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bvgpr"]
Sep 18 15:35:05.745580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:05.745545    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:05.745630 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:05.745603    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:35:06.682460 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:06.682427    2585 generic.go:356] "Generic (PLEG): container finished" podID="6a044247-0dee-476f-b2e4-309e4e0982d8" containerID="71b46e86cbe9d01f349f4621b1915850cc6fae36b6f52ed077b20461377ed8ca" exitCode=0
Sep 18 15:35:06.682825 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:06.682510    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" event={"ID":"6a044247-0dee-476f-b2e4-309e4e0982d8","Type":"ContainerDied","Data":"71b46e86cbe9d01f349f4621b1915850cc6fae36b6f52ed077b20461377ed8ca"}
Sep 18 15:35:07.518071 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:07.518045    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:07.518235 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:07.518081    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:07.518235 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:07.518143    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:35:07.518235 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:07.518210    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:07.518374 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:07.518333    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:35:07.518446 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:07.518418    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:35:09.517594 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:09.517565    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:09.517594 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:09.517587    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:09.518112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:09.517565    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:09.518112 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:09.517720    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4rnz2" podUID="edf901bb-90cc-4841-88db-6de9d72eea52"
Sep 18 15:35:09.518112 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:09.517868    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bvgpr" podUID="f3967c88-e022-44a2-919c-468d1f757435"
Sep 18 15:35:09.518112 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:09.517918    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qq2zh" podUID="f64e3326-8578-4fc8-b8f4-4bf7689a4a61"
Sep 18 15:35:11.153655 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.153616    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-208.ec2.internal" event="NodeReady"
Sep 18 15:35:11.154147 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.153773    2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 15:35:11.197051 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.197020    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6494946bcd-bttdp"]
Sep 18 15:35:11.199682 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.199660    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.203562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.203541    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 15:35:11.203812 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.203788    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 15:35:11.203923 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.203900    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 15:35:11.204139 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.204122    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7tpfr\""
Sep 18 15:35:11.205449 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.205429    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:11.205539 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.205467    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:11.205602 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.205549    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:35:11.205602 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.205560    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 15:35:11.205696 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.205617    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs podName:edf901bb-90cc-4841-88db-6de9d72eea52 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:43.205599088 +0000 UTC m=+67.864391729 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs") pod "network-metrics-daemon-4rnz2" (UID: "edf901bb-90cc-4841-88db-6de9d72eea52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 15:35:11.205696 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.205632    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret podName:f3967c88-e022-44a2-919c-468d1f757435 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:43.205626046 +0000 UTC m=+67.864418683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret") pod "global-pull-secret-syncer-bvgpr" (UID: "f3967c88-e022-44a2-919c-468d1f757435") : object "kube-system"/"original-pull-secret" not registered
Sep 18 15:35:11.213890 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.213864    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mjcsb"]
Sep 18 15:35:11.217679 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.217247    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.220811 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.220788    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 15:35:11.224288 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.224270    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 15:35:11.224377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.224358    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 15:35:11.224502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.224487    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f9wls\""
Sep 18 15:35:11.227222 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.227205    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6494946bcd-bttdp"]
Sep 18 15:35:11.233797 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.233779    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mjcsb"]
Sep 18 15:35:11.246502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.246473    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8vxph"]
Sep 18 15:35:11.248119 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.248101    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8vxph"
Sep 18 15:35:11.253526 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.253505    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 15:35:11.253715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.253703    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 15:35:11.253771 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.253751    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mtwcv\""
Sep 18 15:35:11.253901 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.253889    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 15:35:11.279410 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.279350    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8vxph"]
Sep 18 15:35:11.306794 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.306727    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-installation-pull-secrets\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.306794 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.306764    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6rbn\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-kube-api-access-j6rbn\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.306962 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.306801    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39f429b6-f95f-4ebe-8b50-d31ad9030a07-tmp-dir\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.306962 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.306902    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-certificates\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.306962 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.306943    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9h9v\" (UniqueName: \"kubernetes.io/projected/39f429b6-f95f-4ebe-8b50-d31ad9030a07-kube-api-access-c9h9v\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.306962 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.306960    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-image-registry-private-configuration\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.307125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.306979    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc407841-e310-4671-bd17-fdc0fdd5d38a-ca-trust-extracted\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.307125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.307007    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.307125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.307038    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.307125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.307063    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-bound-sa-token\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.307125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.307090    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/39f429b6-f95f-4ebe-8b50-d31ad9030a07-config-volume\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.307125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.307121    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:11.307345 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.307142    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-trusted-ca\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.307345 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.307237    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 15:35:11.307345 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.307255    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 15:35:11.307345 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.307267    2585 projected.go:196] Error preparing data for projected volume kube-api-access-86zbx for pod openshift-network-diagnostics/network-check-target-qq2zh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:35:11.307345 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.307323    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx podName:f64e3326-8578-4fc8-b8f4-4bf7689a4a61 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:43.307309956 +0000 UTC m=+67.966102594 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-86zbx" (UniqueName: "kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx") pod "network-check-target-qq2zh" (UID: "f64e3326-8578-4fc8-b8f4-4bf7689a4a61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 15:35:11.407587 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.407557    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.407587 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.407589    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/512287cb-bcbb-47c2-b1b7-c4d0d516d227-cert\") pod \"ingress-canary-8vxph\" (UID: \"512287cb-bcbb-47c2-b1b7-c4d0d516d227\") " pod="openshift-ingress-canary/ingress-canary-8vxph"
Sep 18 15:35:11.407800 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.407613    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.407800 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.407636    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-bound-sa-token\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.407800 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.407698    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/39f429b6-f95f-4ebe-8b50-d31ad9030a07-config-volume\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.407800 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.407717    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 15:35:11.407800 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.407732    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6494946bcd-bttdp: secret "image-registry-tls" not found
Sep 18 15:35:11.407800 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.407742    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 15:35:11.407800 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.407771    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-trusted-ca\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.407800 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.407800    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls podName:39f429b6-f95f-4ebe-8b50-d31ad9030a07 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:11.907779934 +0000 UTC m=+36.566572577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls") pod "dns-default-mjcsb" (UID: "39f429b6-f95f-4ebe-8b50-d31ad9030a07") : secret "dns-default-metrics-tls" not found
Sep 18 15:35:11.408240 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.407831    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls podName:dc407841-e310-4671-bd17-fdc0fdd5d38a nodeName:}" failed. No retries permitted until 2026-09-18 15:35:11.907820107 +0000 UTC m=+36.566612750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls") pod "image-registry-6494946bcd-bttdp" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a") : secret "image-registry-tls" not found
Sep 18 15:35:11.408240 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.407905    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-installation-pull-secrets\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.408240 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.407935    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6rbn\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-kube-api-access-j6rbn\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.408240 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.407974    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39f429b6-f95f-4ebe-8b50-d31ad9030a07-tmp-dir\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.408240 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.408003    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-certificates\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.408240 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.408030    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v2tk\" (UniqueName: \"kubernetes.io/projected/512287cb-bcbb-47c2-b1b7-c4d0d516d227-kube-api-access-8v2tk\") pod \"ingress-canary-8vxph\" (UID: \"512287cb-bcbb-47c2-b1b7-c4d0d516d227\") " pod="openshift-ingress-canary/ingress-canary-8vxph"
Sep 18 15:35:11.408240 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.408055    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9h9v\" (UniqueName: \"kubernetes.io/projected/39f429b6-f95f-4ebe-8b50-d31ad9030a07-kube-api-access-c9h9v\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.408240 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.408072    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-image-registry-private-configuration\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.408240 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.408091    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc407841-e310-4671-bd17-fdc0fdd5d38a-ca-trust-extracted\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.408648 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.408273    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39f429b6-f95f-4ebe-8b50-d31ad9030a07-tmp-dir\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.408648 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.408362    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc407841-e310-4671-bd17-fdc0fdd5d38a-ca-trust-extracted\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.408648 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.408394    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/39f429b6-f95f-4ebe-8b50-d31ad9030a07-config-volume\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.408753 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.408658    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-certificates\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.409759 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.409641    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-trusted-ca\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.412338 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.412308    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-installation-pull-secrets\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.412338 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.412330    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-image-registry-private-configuration\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.426503 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.426460    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-bound-sa-token\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.427239 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.427219    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6rbn\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-kube-api-access-j6rbn\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.433491 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.433470    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9h9v\" (UniqueName: \"kubernetes.io/projected/39f429b6-f95f-4ebe-8b50-d31ad9030a07-kube-api-access-c9h9v\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.508879 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.508833    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2tk\" (UniqueName: \"kubernetes.io/projected/512287cb-bcbb-47c2-b1b7-c4d0d516d227-kube-api-access-8v2tk\") pod \"ingress-canary-8vxph\" (UID: \"512287cb-bcbb-47c2-b1b7-c4d0d516d227\") " pod="openshift-ingress-canary/ingress-canary-8vxph"
Sep 18 15:35:11.509063 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.508922    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/512287cb-bcbb-47c2-b1b7-c4d0d516d227-cert\") pod \"ingress-canary-8vxph\" (UID: \"512287cb-bcbb-47c2-b1b7-c4d0d516d227\") " pod="openshift-ingress-canary/ingress-canary-8vxph"
Sep 18 15:35:11.511448 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.511427    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/512287cb-bcbb-47c2-b1b7-c4d0d516d227-cert\") pod \"ingress-canary-8vxph\" (UID: \"512287cb-bcbb-47c2-b1b7-c4d0d516d227\") " pod="openshift-ingress-canary/ingress-canary-8vxph"
Sep 18 15:35:11.518955 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.518929    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:11.519310 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.519295    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:11.519444 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.519423    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v2tk\" (UniqueName: \"kubernetes.io/projected/512287cb-bcbb-47c2-b1b7-c4d0d516d227-kube-api-access-8v2tk\") pod \"ingress-canary-8vxph\" (UID: \"512287cb-bcbb-47c2-b1b7-c4d0d516d227\") " pod="openshift-ingress-canary/ingress-canary-8vxph"
Sep 18 15:35:11.519637 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.519618    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:11.522423 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.522403    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 15:35:11.524737 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.524717    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 15:35:11.524828 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.524761    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 15:35:11.524828 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.524790    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 15:35:11.524968 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.524949    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c572m\""
Sep 18 15:35:11.525098 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.525056    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nqjbs\""
Sep 18 15:35:11.557849 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.557787    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8vxph"
Sep 18 15:35:11.912028 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.911932    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:11.912028 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:11.911985    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:11.912257 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.912121    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 15:35:11.912257 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.912147    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 15:35:11.912257 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.912160    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6494946bcd-bttdp: secret "image-registry-tls" not found
Sep 18 15:35:11.912257 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.912200    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls podName:39f429b6-f95f-4ebe-8b50-d31ad9030a07 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:12.912178022 +0000 UTC m=+37.570970676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls") pod "dns-default-mjcsb" (UID: "39f429b6-f95f-4ebe-8b50-d31ad9030a07") : secret "dns-default-metrics-tls" not found
Sep 18 15:35:11.912257 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:11.912214    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls podName:dc407841-e310-4671-bd17-fdc0fdd5d38a nodeName:}" failed. No retries permitted until 2026-09-18 15:35:12.91220736 +0000 UTC m=+37.570999998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls") pod "image-registry-6494946bcd-bttdp" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a") : secret "image-registry-tls" not found
Sep 18 15:35:12.377344 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:12.377311    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8vxph"]
Sep 18 15:35:12.381368 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:12.381337    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod512287cb_bcbb_47c2_b1b7_c4d0d516d227.slice/crio-ef8e4ebe2857efaee22b92c8c10b9a363aa04f0aa912fa1bf50a680b8c2b8e86 WatchSource:0}: Error finding container ef8e4ebe2857efaee22b92c8c10b9a363aa04f0aa912fa1bf50a680b8c2b8e86: Status 404 returned error can't find the container with id ef8e4ebe2857efaee22b92c8c10b9a363aa04f0aa912fa1bf50a680b8c2b8e86
Sep 18 15:35:12.699173 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:12.699143    2585 generic.go:356] "Generic (PLEG): container finished" podID="6a044247-0dee-476f-b2e4-309e4e0982d8" containerID="fd8f8242e44bac15cb9664042824478a701710c621bfe3769920c6a64d701a50" exitCode=0
Sep 18 15:35:12.699317 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:12.699228    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" event={"ID":"6a044247-0dee-476f-b2e4-309e4e0982d8","Type":"ContainerDied","Data":"fd8f8242e44bac15cb9664042824478a701710c621bfe3769920c6a64d701a50"}
Sep 18 15:35:12.700320 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:12.700302    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8vxph" event={"ID":"512287cb-bcbb-47c2-b1b7-c4d0d516d227","Type":"ContainerStarted","Data":"ef8e4ebe2857efaee22b92c8c10b9a363aa04f0aa912fa1bf50a680b8c2b8e86"}
Sep 18 15:35:12.919106 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:12.919068    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:12.919106 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:12.919111    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:12.919312 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:12.919205    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 15:35:12.919312 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:12.919212    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 15:35:12.919312 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:12.919232    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6494946bcd-bttdp: secret "image-registry-tls" not found
Sep 18 15:35:12.919312 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:12.919276    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls podName:39f429b6-f95f-4ebe-8b50-d31ad9030a07 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:14.919259024 +0000 UTC m=+39.578051662 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls") pod "dns-default-mjcsb" (UID: "39f429b6-f95f-4ebe-8b50-d31ad9030a07") : secret "dns-default-metrics-tls" not found
Sep 18 15:35:12.919452 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:12.919324    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls podName:dc407841-e310-4671-bd17-fdc0fdd5d38a nodeName:}" failed. No retries permitted until 2026-09-18 15:35:14.919315319 +0000 UTC m=+39.578107958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls") pod "image-registry-6494946bcd-bttdp" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a") : secret "image-registry-tls" not found
Sep 18 15:35:13.706569 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:13.706530    2585 generic.go:356] "Generic (PLEG): container finished" podID="6a044247-0dee-476f-b2e4-309e4e0982d8" containerID="08a54f5e7e0734ab7182cebd68eb9e7005e2a52178303d2f42645dd865d04673" exitCode=0
Sep 18 15:35:13.707027 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:13.706592    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" event={"ID":"6a044247-0dee-476f-b2e4-309e4e0982d8","Type":"ContainerDied","Data":"08a54f5e7e0734ab7182cebd68eb9e7005e2a52178303d2f42645dd865d04673"}
Sep 18 15:35:14.714161 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:14.713913    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" event={"ID":"6a044247-0dee-476f-b2e4-309e4e0982d8","Type":"ContainerStarted","Data":"0fbf16d80b732ef4ea8b6d8ead4b9a4eb9804f728109b7619cbb04853333c8d1"}
Sep 18 15:35:14.758554 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:14.758408    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ddhnl" podStartSLOduration=8.271716316 podStartE2EDuration="37.75838563s" totalImagesPullingTime="29.486669314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:14.756182481 +0000 UTC m=+39.414975145" watchObservedRunningTime="2026-09-18 15:35:14.75838563 +0000 UTC m=+39.417178292"
Sep 18 15:35:14.934185 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:14.934105    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:14.934185 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:14.934148    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:14.934393 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:14.934254    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 15:35:14.934393 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:14.934323    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls podName:39f429b6-f95f-4ebe-8b50-d31ad9030a07 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:18.934304039 +0000 UTC m=+43.593096690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls") pod "dns-default-mjcsb" (UID: "39f429b6-f95f-4ebe-8b50-d31ad9030a07") : secret "dns-default-metrics-tls" not found
Sep 18 15:35:14.934393 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:14.934256    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 15:35:14.934393 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:14.934347    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6494946bcd-bttdp: secret "image-registry-tls" not found
Sep 18 15:35:14.934393 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:14.934397    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls podName:dc407841-e310-4671-bd17-fdc0fdd5d38a nodeName:}" failed. No retries permitted until 2026-09-18 15:35:18.934383797 +0000 UTC m=+43.593176435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls") pod "image-registry-6494946bcd-bttdp" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a") : secret "image-registry-tls" not found
Sep 18 15:35:15.718285 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:15.718251    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8vxph" event={"ID":"512287cb-bcbb-47c2-b1b7-c4d0d516d227","Type":"ContainerStarted","Data":"c2a75929981795ebc9b121d67255317e8f06d2f20307fbc6e1ccb3c2a29db2c2"}
Sep 18 15:35:15.749695 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:15.749647    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8vxph" podStartSLOduration=1.902152498 podStartE2EDuration="4.749631264s" totalImagesPullingTime="2.847478766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:15.749158751 +0000 UTC m=+40.407951411" watchObservedRunningTime="2026-09-18 15:35:15.749631264 +0000 UTC m=+40.408423924"
Sep 18 15:35:15.989208 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:15.989178    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ft4rq"]
Sep 18 15:35:16.010472 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.010445    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.013499 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.013474    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ft4rq"]
Sep 18 15:35:16.014164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.014149    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c72vd\""
Sep 18 15:35:16.014290 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.014188    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 15:35:16.014345 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.014288    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 15:35:16.015207 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.015188    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 15:35:16.015207 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.015200    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 15:35:16.143120 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.143087    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9f219a-1495-4ca8-b6c1-863efb7c7140-data-volume\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.143280 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.143133    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.143280 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.143215    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.143280 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.143266    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5c9f219a-1495-4ca8-b6c1-863efb7c7140-crio-socket\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.143406 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.143336    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjqhq\" (UniqueName: \"kubernetes.io/projected/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-api-access-kjqhq\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.243864 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.243771    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9f219a-1495-4ca8-b6c1-863efb7c7140-data-volume\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.243864 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.243821    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.244075 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.243876    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.244075 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.243914    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5c9f219a-1495-4ca8-b6c1-863efb7c7140-crio-socket\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.244075 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.243955    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjqhq\" (UniqueName: \"kubernetes.io/projected/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-api-access-kjqhq\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.244075 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:16.243980    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 15:35:16.244075 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:16.244026    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:16.244075 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:16.244052    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:16.744035141 +0000 UTC m=+41.402827779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : configmap "kube-rbac-proxy" not found
Sep 18 15:35:16.244295 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:16.244091    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:16.74407231 +0000 UTC m=+41.402864963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:16.244295 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.244159    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9f219a-1495-4ca8-b6c1-863efb7c7140-data-volume\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.244295 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.244221    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5c9f219a-1495-4ca8-b6c1-863efb7c7140-crio-socket\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.258994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.258973    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjqhq\" (UniqueName: \"kubernetes.io/projected/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-api-access-kjqhq\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.747432 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.747390    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.747872 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:16.747444    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:16.747872 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:16.747523    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 15:35:16.747872 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:16.747530    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:16.747872 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:16.747617    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:17.747594979 +0000 UTC m=+42.406387637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : configmap "kube-rbac-proxy" not found
Sep 18 15:35:16.747872 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:16.747667    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:17.747653239 +0000 UTC m=+42.406445878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:17.753490 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.753449    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:17.753490 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.753496    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:17.754052 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:17.753586    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:17.754052 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:17.753588    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 15:35:17.754052 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:17.753638    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:19.753624842 +0000 UTC m=+44.412417480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:17.754052 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:17.753650    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:19.753644534 +0000 UTC m=+44.412437172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : configmap "kube-rbac-proxy" not found
Sep 18 15:35:17.861087 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.861051    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-q2878"]
Sep 18 15:35:17.885477 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.885448    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-q2878"]
Sep 18 15:35:17.885661 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.885573    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:17.891797 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.891577    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 18 15:35:17.891797 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.891656    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 18 15:35:17.892711 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.892693    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 18 15:35:17.892861 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.892700    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gl62s\""
Sep 18 15:35:17.892861 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:17.892774    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 18 15:35:18.055823 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.055728    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c49c698c-3b80-4574-a737-adf08cb0547a-signing-cabundle\") pod \"service-ca-55f6fc7b86-q2878\" (UID: \"c49c698c-3b80-4574-a737-adf08cb0547a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.055979 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.055864    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c49c698c-3b80-4574-a737-adf08cb0547a-signing-key\") pod \"service-ca-55f6fc7b86-q2878\" (UID: \"c49c698c-3b80-4574-a737-adf08cb0547a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.055979 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.055954    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dq65\" (UniqueName: \"kubernetes.io/projected/c49c698c-3b80-4574-a737-adf08cb0547a-kube-api-access-5dq65\") pod \"service-ca-55f6fc7b86-q2878\" (UID: \"c49c698c-3b80-4574-a737-adf08cb0547a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.113392 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.113355    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rnzfc_b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1/dns-node-resolver/0.log"
Sep 18 15:35:18.157289 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.157259    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dq65\" (UniqueName: \"kubernetes.io/projected/c49c698c-3b80-4574-a737-adf08cb0547a-kube-api-access-5dq65\") pod \"service-ca-55f6fc7b86-q2878\" (UID: \"c49c698c-3b80-4574-a737-adf08cb0547a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.157415 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.157302    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c49c698c-3b80-4574-a737-adf08cb0547a-signing-cabundle\") pod \"service-ca-55f6fc7b86-q2878\" (UID: \"c49c698c-3b80-4574-a737-adf08cb0547a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.157415 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.157381    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c49c698c-3b80-4574-a737-adf08cb0547a-signing-key\") pod \"service-ca-55f6fc7b86-q2878\" (UID: \"c49c698c-3b80-4574-a737-adf08cb0547a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.158055 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.158034    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c49c698c-3b80-4574-a737-adf08cb0547a-signing-cabundle\") pod \"service-ca-55f6fc7b86-q2878\" (UID: \"c49c698c-3b80-4574-a737-adf08cb0547a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.159683 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.159662    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c49c698c-3b80-4574-a737-adf08cb0547a-signing-key\") pod \"service-ca-55f6fc7b86-q2878\" (UID: \"c49c698c-3b80-4574-a737-adf08cb0547a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.165908 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.165888    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dq65\" (UniqueName: \"kubernetes.io/projected/c49c698c-3b80-4574-a737-adf08cb0547a-kube-api-access-5dq65\") pod \"service-ca-55f6fc7b86-q2878\" (UID: \"c49c698c-3b80-4574-a737-adf08cb0547a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.193852 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.193808    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-q2878"
Sep 18 15:35:18.491510 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.491483    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-q2878"]
Sep 18 15:35:18.494394 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:18.494369    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc49c698c_3b80_4574_a737_adf08cb0547a.slice/crio-cbfbfcc41072d35ecc28165a3a9a3b4b8bc9d373e04da3c40f5c3ae009270d56 WatchSource:0}: Error finding container cbfbfcc41072d35ecc28165a3a9a3b4b8bc9d373e04da3c40f5c3ae009270d56: Status 404 returned error can't find the container with id cbfbfcc41072d35ecc28165a3a9a3b4b8bc9d373e04da3c40f5c3ae009270d56
Sep 18 15:35:18.719972 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.719947    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hhm7w_9eb2cbd7-9c45-4158-88f1-39cc90d66fc4/node-ca/0.log"
Sep 18 15:35:18.726799 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.726774    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-q2878" event={"ID":"c49c698c-3b80-4574-a737-adf08cb0547a","Type":"ContainerStarted","Data":"cbfbfcc41072d35ecc28165a3a9a3b4b8bc9d373e04da3c40f5c3ae009270d56"}
Sep 18 15:35:18.966343 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.966249    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:18.966343 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:18.966295    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:18.966812 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:18.966411    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 15:35:18.966812 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:18.966425    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 15:35:18.966812 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:18.966449    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6494946bcd-bttdp: secret "image-registry-tls" not found
Sep 18 15:35:18.966812 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:18.966499    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls podName:39f429b6-f95f-4ebe-8b50-d31ad9030a07 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:26.96648133 +0000 UTC m=+51.625273968 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls") pod "dns-default-mjcsb" (UID: "39f429b6-f95f-4ebe-8b50-d31ad9030a07") : secret "dns-default-metrics-tls" not found
Sep 18 15:35:18.966812 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:18.966513    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls podName:dc407841-e310-4671-bd17-fdc0fdd5d38a nodeName:}" failed. No retries permitted until 2026-09-18 15:35:26.966507572 +0000 UTC m=+51.625300209 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls") pod "image-registry-6494946bcd-bttdp" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a") : secret "image-registry-tls" not found
Sep 18 15:35:19.511256 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:19.511222    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8vxph_512287cb-bcbb-47c2-b1b7-c4d0d516d227/serve-healthcheck-canary/0.log"
Sep 18 15:35:19.774010 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:19.773939    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:19.774010 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:19.773985    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:19.774200 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:19.774062    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 15:35:19.774200 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:19.774078    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:19.774200 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:19.774130    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:23.774112228 +0000 UTC m=+48.432904865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : configmap "kube-rbac-proxy" not found
Sep 18 15:35:19.774200 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:19.774144    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:23.774137637 +0000 UTC m=+48.432930275 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:21.736078 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:21.736042    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-q2878" event={"ID":"c49c698c-3b80-4574-a737-adf08cb0547a","Type":"ContainerStarted","Data":"a5685314138905727bf5371d3823b151ed550aeab0bf1d03b0ae7d3446127ce1"}
Sep 18 15:35:21.766058 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:21.765998    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-q2878" podStartSLOduration=2.564436122 podStartE2EDuration="4.76597888s" totalImagesPullingTime="2.201542758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:21.764147463 +0000 UTC m=+46.422940123" watchObservedRunningTime="2026-09-18 15:35:21.76597888 +0000 UTC m=+46.424771541"
Sep 18 15:35:23.808918 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:23.808873    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:23.809316 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:23.808926    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:23.809316 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:23.809002    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 15:35:23.809316 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:23.809059    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:23.809316 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:23.809073    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:31.809053757 +0000 UTC m=+56.467846415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : configmap "kube-rbac-proxy" not found
Sep 18 15:35:23.809316 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:23.809095    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:31.809085236 +0000 UTC m=+56.467877878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:27.032747 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.032711    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:27.032747 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.032751    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:27.035056 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.035028    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39f429b6-f95f-4ebe-8b50-d31ad9030a07-metrics-tls\") pod \"dns-default-mjcsb\" (UID: \"39f429b6-f95f-4ebe-8b50-d31ad9030a07\") " pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:27.035278 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.035257    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") pod \"image-registry-6494946bcd-bttdp\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") " pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:27.111191 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.111160    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:27.127487 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.127464    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:27.422629 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.422606    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6494946bcd-bttdp"]
Sep 18 15:35:27.425767 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:27.425740    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc407841_e310_4671_bd17_fdc0fdd5d38a.slice/crio-0a4e26c2adfa9354a06109e91c161d8137fe41a8ce24c8c32bbe08529271f145 WatchSource:0}: Error finding container 0a4e26c2adfa9354a06109e91c161d8137fe41a8ce24c8c32bbe08529271f145: Status 404 returned error can't find the container with id 0a4e26c2adfa9354a06109e91c161d8137fe41a8ce24c8c32bbe08529271f145
Sep 18 15:35:27.447220 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.447197    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mjcsb"]
Sep 18 15:35:27.449672 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:27.449651    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39f429b6_f95f_4ebe_8b50_d31ad9030a07.slice/crio-7cd0ead025924f2bfc3cdec59229c7c0a9b542eac970e0fb4636c7193be3eb13 WatchSource:0}: Error finding container 7cd0ead025924f2bfc3cdec59229c7c0a9b542eac970e0fb4636c7193be3eb13: Status 404 returned error can't find the container with id 7cd0ead025924f2bfc3cdec59229c7c0a9b542eac970e0fb4636c7193be3eb13
Sep 18 15:35:27.752288 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.752257    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mjcsb" event={"ID":"39f429b6-f95f-4ebe-8b50-d31ad9030a07","Type":"ContainerStarted","Data":"7cd0ead025924f2bfc3cdec59229c7c0a9b542eac970e0fb4636c7193be3eb13"}
Sep 18 15:35:27.753815 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.753793    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6494946bcd-bttdp" event={"ID":"dc407841-e310-4671-bd17-fdc0fdd5d38a","Type":"ContainerStarted","Data":"4d2e468ad7e6a78a0e7ec217f007d104541f079bc5c3f36626b0f555f43711ee"}
Sep 18 15:35:27.753927 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.753827    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6494946bcd-bttdp" event={"ID":"dc407841-e310-4671-bd17-fdc0fdd5d38a","Type":"ContainerStarted","Data":"0a4e26c2adfa9354a06109e91c161d8137fe41a8ce24c8c32bbe08529271f145"}
Sep 18 15:35:27.753966 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.753930    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:27.779136 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:27.779089    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6494946bcd-bttdp" podStartSLOduration=20.779075062 podStartE2EDuration="20.779075062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:27.77852405 +0000 UTC m=+52.437316711" watchObservedRunningTime="2026-09-18 15:35:27.779075062 +0000 UTC m=+52.437867721"
Sep 18 15:35:30.764196 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:30.764159    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mjcsb" event={"ID":"39f429b6-f95f-4ebe-8b50-d31ad9030a07","Type":"ContainerStarted","Data":"dc2adf23ae1f1ad6d58165d37284531b03f91b6b35c411e8c1f434e857dec420"}
Sep 18 15:35:30.764543 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:30.764216    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mjcsb" event={"ID":"39f429b6-f95f-4ebe-8b50-d31ad9030a07","Type":"ContainerStarted","Data":"11222e7a4439db5b8e96a93f971f3058c4011147b903266789a8c8afd8c6c729"}
Sep 18 15:35:30.764543 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:30.764284    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:30.782065 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:30.782022    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mjcsb" podStartSLOduration=17.301784823 podStartE2EDuration="19.7820084s" totalImagesPullingTime="2.480223577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:30.781573122 +0000 UTC m=+55.440365782" watchObservedRunningTime="2026-09-18 15:35:30.7820084 +0000 UTC m=+55.440801059"
Sep 18 15:35:31.872156 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:31.872120    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:31.872598 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:31.872172    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:31.872598 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:31.872295    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:31.872598 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:31.872375    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls podName:5c9f219a-1495-4ca8-b6c1-863efb7c7140 nodeName:}" failed. No retries permitted until 2026-09-18 15:35:47.872353091 +0000 UTC m=+72.531145739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ft4rq" (UID: "5c9f219a-1495-4ca8-b6c1-863efb7c7140") : secret "insights-runtime-extractor-tls" not found
Sep 18 15:35:31.872709 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:31.872620    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c9f219a-1495-4ca8-b6c1-863efb7c7140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:37.694829 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:37.694799    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6wqf6"
Sep 18 15:35:40.600258 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.600224    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"]
Sep 18 15:35:40.603038 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.603016    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"]
Sep 18 15:35:40.603170 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.603134    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"
Sep 18 15:35:40.604762 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.604745    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.615239 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.615221    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 15:35:40.615335 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.615245    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 15:35:40.616488 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.616468    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 18 15:35:40.616574 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.616476    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 15:35:40.616574 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.616508    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 18 15:35:40.616574 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.616521    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-mdrng\""
Sep 18 15:35:40.620110 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.620089    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"]
Sep 18 15:35:40.630692 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.630674    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f76vp\" (UniqueName: \"kubernetes.io/projected/3935c571-da5a-4c44-9bea-e90a5ac15642-kube-api-access-f76vp\") pod \"klusterlet-addon-workmgr-85c9dfdbf8-dgml2\" (UID: \"3935c571-da5a-4c44-9bea-e90a5ac15642\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.630777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.630710    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3935c571-da5a-4c44-9bea-e90a5ac15642-klusterlet-config\") pod \"klusterlet-addon-workmgr-85c9dfdbf8-dgml2\" (UID: \"3935c571-da5a-4c44-9bea-e90a5ac15642\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.630777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.630732    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7035d916-a973-49ad-90f0-b2d7d4f626d9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c485ccf56-j8tm2\" (UID: \"7035d916-a973-49ad-90f0-b2d7d4f626d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"
Sep 18 15:35:40.630777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.630754    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25kw9\" (UniqueName: \"kubernetes.io/projected/7035d916-a973-49ad-90f0-b2d7d4f626d9-kube-api-access-25kw9\") pod \"managed-serviceaccount-addon-agent-5c485ccf56-j8tm2\" (UID: \"7035d916-a973-49ad-90f0-b2d7d4f626d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"
Sep 18 15:35:40.630777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.630769    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3935c571-da5a-4c44-9bea-e90a5ac15642-tmp\") pod \"klusterlet-addon-workmgr-85c9dfdbf8-dgml2\" (UID: \"3935c571-da5a-4c44-9bea-e90a5ac15642\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.632455 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.632433    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6494946bcd-bttdp"]
Sep 18 15:35:40.635248 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.635223    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"]
Sep 18 15:35:40.731313 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.731270    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7035d916-a973-49ad-90f0-b2d7d4f626d9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c485ccf56-j8tm2\" (UID: \"7035d916-a973-49ad-90f0-b2d7d4f626d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"
Sep 18 15:35:40.731519 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.731328    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25kw9\" (UniqueName: \"kubernetes.io/projected/7035d916-a973-49ad-90f0-b2d7d4f626d9-kube-api-access-25kw9\") pod \"managed-serviceaccount-addon-agent-5c485ccf56-j8tm2\" (UID: \"7035d916-a973-49ad-90f0-b2d7d4f626d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"
Sep 18 15:35:40.731519 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.731347    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3935c571-da5a-4c44-9bea-e90a5ac15642-tmp\") pod \"klusterlet-addon-workmgr-85c9dfdbf8-dgml2\" (UID: \"3935c571-da5a-4c44-9bea-e90a5ac15642\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.731519 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.731411    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f76vp\" (UniqueName: \"kubernetes.io/projected/3935c571-da5a-4c44-9bea-e90a5ac15642-kube-api-access-f76vp\") pod \"klusterlet-addon-workmgr-85c9dfdbf8-dgml2\" (UID: \"3935c571-da5a-4c44-9bea-e90a5ac15642\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.731519 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.731447    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3935c571-da5a-4c44-9bea-e90a5ac15642-klusterlet-config\") pod \"klusterlet-addon-workmgr-85c9dfdbf8-dgml2\" (UID: \"3935c571-da5a-4c44-9bea-e90a5ac15642\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.731801 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.731763    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3935c571-da5a-4c44-9bea-e90a5ac15642-tmp\") pod \"klusterlet-addon-workmgr-85c9dfdbf8-dgml2\" (UID: \"3935c571-da5a-4c44-9bea-e90a5ac15642\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.733947 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.733925    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3935c571-da5a-4c44-9bea-e90a5ac15642-klusterlet-config\") pod \"klusterlet-addon-workmgr-85c9dfdbf8-dgml2\" (UID: \"3935c571-da5a-4c44-9bea-e90a5ac15642\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.734041 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.733951    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7035d916-a973-49ad-90f0-b2d7d4f626d9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c485ccf56-j8tm2\" (UID: \"7035d916-a973-49ad-90f0-b2d7d4f626d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"
Sep 18 15:35:40.744029 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.744006    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f76vp\" (UniqueName: \"kubernetes.io/projected/3935c571-da5a-4c44-9bea-e90a5ac15642-kube-api-access-f76vp\") pod \"klusterlet-addon-workmgr-85c9dfdbf8-dgml2\" (UID: \"3935c571-da5a-4c44-9bea-e90a5ac15642\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:40.746111 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.746093    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25kw9\" (UniqueName: \"kubernetes.io/projected/7035d916-a973-49ad-90f0-b2d7d4f626d9-kube-api-access-25kw9\") pod \"managed-serviceaccount-addon-agent-5c485ccf56-j8tm2\" (UID: \"7035d916-a973-49ad-90f0-b2d7d4f626d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"
Sep 18 15:35:40.772223 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.772202    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mjcsb"
Sep 18 15:35:40.923127 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.923049    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"
Sep 18 15:35:40.928580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:40.928559    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:41.192544 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:41.192463    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2"]
Sep 18 15:35:41.195164 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:41.195141    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7035d916_a973_49ad_90f0_b2d7d4f626d9.slice/crio-bf2c7b75f9db248f5d3fae3b245229b72238eebd9cd8477bb7956c4835d415db WatchSource:0}: Error finding container bf2c7b75f9db248f5d3fae3b245229b72238eebd9cd8477bb7956c4835d415db: Status 404 returned error can't find the container with id bf2c7b75f9db248f5d3fae3b245229b72238eebd9cd8477bb7956c4835d415db
Sep 18 15:35:41.199986 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:41.199967    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"]
Sep 18 15:35:41.202462 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:41.202430    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3935c571_da5a_4c44_9bea_e90a5ac15642.slice/crio-84fe5c32cf31da66653393fe344dd6e6385f28f4d978f6eddace4a9b5b85b479 WatchSource:0}: Error finding container 84fe5c32cf31da66653393fe344dd6e6385f28f4d978f6eddace4a9b5b85b479: Status 404 returned error can't find the container with id 84fe5c32cf31da66653393fe344dd6e6385f28f4d978f6eddace4a9b5b85b479
Sep 18 15:35:41.802208 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:41.802173    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2" event={"ID":"3935c571-da5a-4c44-9bea-e90a5ac15642","Type":"ContainerStarted","Data":"84fe5c32cf31da66653393fe344dd6e6385f28f4d978f6eddace4a9b5b85b479"}
Sep 18 15:35:41.803219 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:41.803196    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2" event={"ID":"7035d916-a973-49ad-90f0-b2d7d4f626d9","Type":"ContainerStarted","Data":"bf2c7b75f9db248f5d3fae3b245229b72238eebd9cd8477bb7956c4835d415db"}
Sep 18 15:35:42.754603 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:42.754181    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"]
Sep 18 15:35:42.758103 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:42.758075    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"
Sep 18 15:35:42.761795 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:42.761771    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-g6dzw\""
Sep 18 15:35:42.762034 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:42.762018    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 18 15:35:42.768278 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:42.768253    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"]
Sep 18 15:35:42.848503 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:42.848339    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e6105c12-ac05-4aa8-b9d2-2ec4f4ff723a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-8cbfj\" (UID: \"e6105c12-ac05-4aa8-b9d2-2ec4f4ff723a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"
Sep 18 15:35:42.949767 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:42.949230    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e6105c12-ac05-4aa8-b9d2-2ec4f4ff723a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-8cbfj\" (UID: \"e6105c12-ac05-4aa8-b9d2-2ec4f4ff723a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"
Sep 18 15:35:42.956112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:42.956051    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e6105c12-ac05-4aa8-b9d2-2ec4f4ff723a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-8cbfj\" (UID: \"e6105c12-ac05-4aa8-b9d2-2ec4f4ff723a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"
Sep 18 15:35:43.071827 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.071690    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"
Sep 18 15:35:43.252548 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.252514    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:43.252744 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.252606    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:43.255668 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.255643    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 15:35:43.255801 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.255679    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 15:35:43.265940 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.265904    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf901bb-90cc-4841-88db-6de9d72eea52-metrics-certs\") pod \"network-metrics-daemon-4rnz2\" (UID: \"edf901bb-90cc-4841-88db-6de9d72eea52\") " pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:43.266056 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.265963    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3967c88-e022-44a2-919c-468d1f757435-original-pull-secret\") pod \"global-pull-secret-syncer-bvgpr\" (UID: \"f3967c88-e022-44a2-919c-468d1f757435\") " pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:43.341289 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.341190    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c572m\""
Sep 18 15:35:43.343307 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.343280    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bvgpr"
Sep 18 15:35:43.348748 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.348640    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4rnz2"
Sep 18 15:35:43.354102 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.354075    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:43.356997 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.356969    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 15:35:43.367211 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.367188    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 15:35:43.372918 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.372618    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"]
Sep 18 15:35:43.377997 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.377973    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86zbx\" (UniqueName: \"kubernetes.io/projected/f64e3326-8578-4fc8-b8f4-4bf7689a4a61-kube-api-access-86zbx\") pod \"network-check-target-qq2zh\" (UID: \"f64e3326-8578-4fc8-b8f4-4bf7689a4a61\") " pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:43.634385 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.634315    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nqjbs\""
Sep 18 15:35:43.641297 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:43.641263    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:44.836248 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:44.836218    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6105c12_ac05_4aa8_b9d2_2ec4f4ff723a.slice/crio-94d448f2b5fcc0b4ab1af6178c1d26aecc74e35472a8bf36918eba853c642f62 WatchSource:0}: Error finding container 94d448f2b5fcc0b4ab1af6178c1d26aecc74e35472a8bf36918eba853c642f62: Status 404 returned error can't find the container with id 94d448f2b5fcc0b4ab1af6178c1d26aecc74e35472a8bf36918eba853c642f62
Sep 18 15:35:45.820301 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:45.820257    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj" event={"ID":"e6105c12-ac05-4aa8-b9d2-2ec4f4ff723a","Type":"ContainerStarted","Data":"94d448f2b5fcc0b4ab1af6178c1d26aecc74e35472a8bf36918eba853c642f62"}
Sep 18 15:35:46.710136 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.710110    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qq2zh"]
Sep 18 15:35:46.712720 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:46.712658    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64e3326_8578_4fc8_b8f4_4bf7689a4a61.slice/crio-7afa63b9580dff4b409cea64e1920443052be01541710430809b5e1123325330 WatchSource:0}: Error finding container 7afa63b9580dff4b409cea64e1920443052be01541710430809b5e1123325330: Status 404 returned error can't find the container with id 7afa63b9580dff4b409cea64e1920443052be01541710430809b5e1123325330
Sep 18 15:35:46.713686 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.713622    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4rnz2"]
Sep 18 15:35:46.714696 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.714677    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bvgpr"]
Sep 18 15:35:46.716218 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:46.716194    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3967c88_e022_44a2_919c_468d1f757435.slice/crio-097d12fea6d818cd9ce00a14b2837c0bc4d419a4c2364685c4df2e8829b870f2 WatchSource:0}: Error finding container 097d12fea6d818cd9ce00a14b2837c0bc4d419a4c2364685c4df2e8829b870f2: Status 404 returned error can't find the container with id 097d12fea6d818cd9ce00a14b2837c0bc4d419a4c2364685c4df2e8829b870f2
Sep 18 15:35:46.716919 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:46.716900    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedf901bb_90cc_4841_88db_6de9d72eea52.slice/crio-b93246733b743f30e4a03c07769b12ab299d96f3cc075f94d7135283e9e4c2f1 WatchSource:0}: Error finding container b93246733b743f30e4a03c07769b12ab299d96f3cc075f94d7135283e9e4c2f1: Status 404 returned error can't find the container with id b93246733b743f30e4a03c07769b12ab299d96f3cc075f94d7135283e9e4c2f1
Sep 18 15:35:46.826462 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.826317    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2" event={"ID":"3935c571-da5a-4c44-9bea-e90a5ac15642","Type":"ContainerStarted","Data":"cbff2ca76eaba15d2349ffa2b676fea26bf25c33d67e623bd6a35fa727330710"}
Sep 18 15:35:46.826637 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.826534    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:46.828396 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.828354    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2" event={"ID":"7035d916-a973-49ad-90f0-b2d7d4f626d9","Type":"ContainerStarted","Data":"956d7dbd531402db66087f82775d8f329341758281a673136033e90391cf17d3"}
Sep 18 15:35:46.828979 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.828959    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2"
Sep 18 15:35:46.830015 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.829990    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bvgpr" event={"ID":"f3967c88-e022-44a2-919c-468d1f757435","Type":"ContainerStarted","Data":"097d12fea6d818cd9ce00a14b2837c0bc4d419a4c2364685c4df2e8829b870f2"}
Sep 18 15:35:46.831389 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.831367    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qq2zh" event={"ID":"f64e3326-8578-4fc8-b8f4-4bf7689a4a61","Type":"ContainerStarted","Data":"7afa63b9580dff4b409cea64e1920443052be01541710430809b5e1123325330"}
Sep 18 15:35:46.832937 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.832917    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4rnz2" event={"ID":"edf901bb-90cc-4841-88db-6de9d72eea52","Type":"ContainerStarted","Data":"b93246733b743f30e4a03c07769b12ab299d96f3cc075f94d7135283e9e4c2f1"}
Sep 18 15:35:46.843120 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.843067    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85c9dfdbf8-dgml2" podStartSLOduration=1.588152541 podStartE2EDuration="6.843055355s" totalImagesPullingTime="5.254902814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:46.842404691 +0000 UTC m=+71.501197352" watchObservedRunningTime="2026-09-18 15:35:46.843055355 +0000 UTC m=+71.501848014"
Sep 18 15:35:46.879921 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.879867    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c485ccf56-j8tm2" podStartSLOduration=1.6361643259999998 podStartE2EDuration="6.879825614s" totalImagesPullingTime="5.243661288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:46.879698481 +0000 UTC m=+71.538491142" watchObservedRunningTime="2026-09-18 15:35:46.879825614 +0000 UTC m=+71.538618274"
Sep 18 15:35:46.903742 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.903715    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-594f8d6749-gglqm"]
Sep 18 15:35:46.905815 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.905792    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:46.908359 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.908337    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8fq62\""
Sep 18 15:35:46.908494 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.908458    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 15:35:46.909132 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.908697    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 15:35:46.909132 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.908865    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 15:35:46.909132 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.908945    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 15:35:46.909132 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.909036    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 15:35:46.909434 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.909136    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 15:35:46.909555 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.909528    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 15:35:46.915003 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.914975    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 15:35:46.916121 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.916100    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-594f8d6749-gglqm"]
Sep 18 15:35:46.984978 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.984947    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-oauth-serving-cert\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:46.985146 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.984995    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-config\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:46.985146 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.985047    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-serving-cert\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:46.985265 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.985154    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-service-ca\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:46.985265 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.985211    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-oauth-config\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:46.985265 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.985244    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-trusted-ca-bundle\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:46.985395 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:46.985281    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb6qb\" (UniqueName: \"kubernetes.io/projected/70120c62-b291-40ff-83ef-eb60ea61f8dc-kube-api-access-wb6qb\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.086015 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.085978    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-oauth-config\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.086015 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.086021    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-trusted-ca-bundle\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.086247 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.086052    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb6qb\" (UniqueName: \"kubernetes.io/projected/70120c62-b291-40ff-83ef-eb60ea61f8dc-kube-api-access-wb6qb\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.086247 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.086082    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-oauth-serving-cert\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.086247 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.086102    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-config\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.086247 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.086157    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-serving-cert\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.086247 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.086207    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-service-ca\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.087016 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.086965    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-service-ca\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.087250 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.087180    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-config\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.087395 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.087277    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-oauth-serving-cert\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.087395 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.087361    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-trusted-ca-bundle\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.089001 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.088978    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-serving-cert\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.089099 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.089048    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-oauth-config\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.094951 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.094932    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb6qb\" (UniqueName: \"kubernetes.io/projected/70120c62-b291-40ff-83ef-eb60ea61f8dc-kube-api-access-wb6qb\") pod \"console-594f8d6749-gglqm\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") " pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.217237 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.217205    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:47.508293 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.508258    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-594f8d6749-gglqm"]
Sep 18 15:35:47.515110 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:47.515081    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70120c62_b291_40ff_83ef_eb60ea61f8dc.slice/crio-7b89586677d44f2b47e61f15dff139a6481ff2fc3fc104500cdc183c9366e324 WatchSource:0}: Error finding container 7b89586677d44f2b47e61f15dff139a6481ff2fc3fc104500cdc183c9366e324: Status 404 returned error can't find the container with id 7b89586677d44f2b47e61f15dff139a6481ff2fc3fc104500cdc183c9366e324
Sep 18 15:35:47.839343 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.839308    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj" event={"ID":"e6105c12-ac05-4aa8-b9d2-2ec4f4ff723a","Type":"ContainerStarted","Data":"ff01e599c46228b0e5e42c135d66289140a9ad2e869bf4ca0b37a37582afd20e"}
Sep 18 15:35:47.839875 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.839453    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"
Sep 18 15:35:47.841248 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.841219    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f8d6749-gglqm" event={"ID":"70120c62-b291-40ff-83ef-eb60ea61f8dc","Type":"ContainerStarted","Data":"7b89586677d44f2b47e61f15dff139a6481ff2fc3fc104500cdc183c9366e324"}
Sep 18 15:35:47.846159 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.846116    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj"
Sep 18 15:35:47.856583 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.856532    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-8cbfj" podStartSLOduration=3.569227961 podStartE2EDuration="5.856517891s" totalImagesPullingTime="2.28728993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:47.855968944 +0000 UTC m=+72.514761604" watchObservedRunningTime="2026-09-18 15:35:47.856517891 +0000 UTC m=+72.515310555"
Sep 18 15:35:47.892788 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.892755    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:47.897365 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:47.897319    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c9f219a-1495-4ca8-b6c1-863efb7c7140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ft4rq\" (UID: \"5c9f219a-1495-4ca8-b6c1-863efb7c7140\") " pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:48.123303 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.123110    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c72vd\""
Sep 18 15:35:48.131531 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.131484    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ft4rq"
Sep 18 15:35:48.443825 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.443611    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ft4rq"]
Sep 18 15:35:48.449328 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:48.449290    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c9f219a_1495_4ca8_b6c1_863efb7c7140.slice/crio-12b5863de094fa71cbbca0578bbfa6bfdba0f7c0c1888c562fd31dda24f26233 WatchSource:0}: Error finding container 12b5863de094fa71cbbca0578bbfa6bfdba0f7c0c1888c562fd31dda24f26233: Status 404 returned error can't find the container with id 12b5863de094fa71cbbca0578bbfa6bfdba0f7c0c1888c562fd31dda24f26233
Sep 18 15:35:48.834341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.833099    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"]
Sep 18 15:35:48.838052 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.836697    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:48.842704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.841556    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 18 15:35:48.842704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.841794    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rchx5\""
Sep 18 15:35:48.842704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.842158    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 18 15:35:48.842704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.842350    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 15:35:48.842704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.842538    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 15:35:48.848626 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.847302    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"]
Sep 18 15:35:48.848626 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.847484    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 15:35:48.851142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.851011    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ft4rq" event={"ID":"5c9f219a-1495-4ca8-b6c1-863efb7c7140","Type":"ContainerStarted","Data":"95feac9569eea33c7846690ba145f418264e341dea7fb48bdc2ccf5d9f1c1b00"}
Sep 18 15:35:48.851142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.851093    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ft4rq" event={"ID":"5c9f219a-1495-4ca8-b6c1-863efb7c7140","Type":"ContainerStarted","Data":"12b5863de094fa71cbbca0578bbfa6bfdba0f7c0c1888c562fd31dda24f26233"}
Sep 18 15:35:48.863880 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.863351    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4rnz2" event={"ID":"edf901bb-90cc-4841-88db-6de9d72eea52","Type":"ContainerStarted","Data":"ef058b2b1b3be37c68f2bb294c04c70d7090a005810f587dc84bfc7265b382f6"}
Sep 18 15:35:48.863880 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.863405    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4rnz2" event={"ID":"edf901bb-90cc-4841-88db-6de9d72eea52","Type":"ContainerStarted","Data":"0dc7d7ae47500ec14bdcf71fdbd91b20fa579207c22eb5aee161f90dd9841a2a"}
Sep 18 15:35:48.902131 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.900250    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2108e61-95bf-4790-8073-7254ded185d6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:48.902131 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.900306    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2108e61-95bf-4790-8073-7254ded185d6-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:48.902131 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.900369    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2108e61-95bf-4790-8073-7254ded185d6-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:48.902131 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:48.900397    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98xbn\" (UniqueName: \"kubernetes.io/projected/a2108e61-95bf-4790-8073-7254ded185d6-kube-api-access-98xbn\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.002677 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.001766    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2108e61-95bf-4790-8073-7254ded185d6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.002677 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.001831    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2108e61-95bf-4790-8073-7254ded185d6-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.002677 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.001883    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2108e61-95bf-4790-8073-7254ded185d6-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.002677 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.001912    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98xbn\" (UniqueName: \"kubernetes.io/projected/a2108e61-95bf-4790-8073-7254ded185d6-kube-api-access-98xbn\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.003348 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.003097    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2108e61-95bf-4790-8073-7254ded185d6-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.012451 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.012349    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2108e61-95bf-4790-8073-7254ded185d6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.020967 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.020825    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2108e61-95bf-4790-8073-7254ded185d6-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.026038 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.025992    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98xbn\" (UniqueName: \"kubernetes.io/projected/a2108e61-95bf-4790-8073-7254ded185d6-kube-api-access-98xbn\") pod \"prometheus-operator-7bd5d56d6-rmq6h\" (UID: \"a2108e61-95bf-4790-8073-7254ded185d6\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.157649 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.157333    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"
Sep 18 15:35:49.460143 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.459067    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4rnz2" podStartSLOduration=71.308306012 podStartE2EDuration="1m12.459044979s" totalImagesPullingTime="1.150738967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:48.88097228 +0000 UTC m=+73.539764941" watchObservedRunningTime="2026-09-18 15:35:49.459044979 +0000 UTC m=+74.117837641"
Sep 18 15:35:49.460143 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.459987    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h"]
Sep 18 15:35:49.485396 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:49.485362    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2108e61_95bf_4790_8073_7254ded185d6.slice/crio-a4d608612ec4e78b3cbd819a69a439929980fdb9c64177a8e753314c043d57a1 WatchSource:0}: Error finding container a4d608612ec4e78b3cbd819a69a439929980fdb9c64177a8e753314c043d57a1: Status 404 returned error can't find the container with id a4d608612ec4e78b3cbd819a69a439929980fdb9c64177a8e753314c043d57a1
Sep 18 15:35:49.868007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:49.867970    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h" event={"ID":"a2108e61-95bf-4790-8073-7254ded185d6","Type":"ContainerStarted","Data":"a4d608612ec4e78b3cbd819a69a439929980fdb9c64177a8e753314c043d57a1"}
Sep 18 15:35:50.638200 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:50.638173    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:35:55.892678 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.892623    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bvgpr" event={"ID":"f3967c88-e022-44a2-919c-468d1f757435","Type":"ContainerStarted","Data":"e3d8e535a80a3c9e08f43406c1238ff35dcc606f0e55077eee21a17adc0c0b67"}
Sep 18 15:35:55.894577 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.894462    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qq2zh" event={"ID":"f64e3326-8578-4fc8-b8f4-4bf7689a4a61","Type":"ContainerStarted","Data":"6c883823ca5e38c7d491d5b1d3b2c812aa0af8d75ff163758204d10f02f0d5bf"}
Sep 18 15:35:55.894577 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.894556    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:35:55.896307 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.896280    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f8d6749-gglqm" event={"ID":"70120c62-b291-40ff-83ef-eb60ea61f8dc","Type":"ContainerStarted","Data":"ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b"}
Sep 18 15:35:55.898338 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.898312    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ft4rq" event={"ID":"5c9f219a-1495-4ca8-b6c1-863efb7c7140","Type":"ContainerStarted","Data":"bd9bf1733e608b27bda29ee72656367f90ceee87e2585d91d748396dc3548ace"}
Sep 18 15:35:55.900227 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.900203    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h" event={"ID":"a2108e61-95bf-4790-8073-7254ded185d6","Type":"ContainerStarted","Data":"b4ba7e60cfc061eb3729d4a919aa9d5923781aa8881e878cafec8b6e674d64fa"}
Sep 18 15:35:55.900324 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.900250    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h" event={"ID":"a2108e61-95bf-4790-8073-7254ded185d6","Type":"ContainerStarted","Data":"70381d7ed10eb95726ccfd0f2d099c3c19f061d1fc63f971ad0b19b7e10768ad"}
Sep 18 15:35:55.927469 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.927426    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-rmq6h" podStartSLOduration=2.444008058 podStartE2EDuration="7.927412658s" totalImagesPullingTime="5.4834046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:55.927002459 +0000 UTC m=+80.585795119" watchObservedRunningTime="2026-09-18 15:35:55.927412658 +0000 UTC m=+80.586205315"
Sep 18 15:35:55.927601 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.927504    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bvgpr" podStartSLOduration=70.676009211 podStartE2EDuration="1m18.927497908s" totalImagesPullingTime="8.251488697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:55.907888599 +0000 UTC m=+80.566681262" watchObservedRunningTime="2026-09-18 15:35:55.927497908 +0000 UTC m=+80.586290568"
Sep 18 15:35:55.972776 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.972719    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-594f8d6749-gglqm" podStartSLOduration=2.498233995 podStartE2EDuration="9.972699518s" totalImagesPullingTime="7.474465523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:55.951788314 +0000 UTC m=+80.610580975" watchObservedRunningTime="2026-09-18 15:35:55.972699518 +0000 UTC m=+80.631492179"
Sep 18 15:35:55.973073 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:55.973044    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qq2zh" podStartSLOduration=70.765581136 podStartE2EDuration="1m18.973032613s" totalImagesPullingTime="8.207451477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:55.971702467 +0000 UTC m=+80.630495128" watchObservedRunningTime="2026-09-18 15:35:55.973032613 +0000 UTC m=+80.631825274"
Sep 18 15:35:56.905492 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:56.905452    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ft4rq" event={"ID":"5c9f219a-1495-4ca8-b6c1-863efb7c7140","Type":"ContainerStarted","Data":"adeb5df28095e01575a61c5738174f8fa48704ebc9adfb2536039530f0caf303"}
Sep 18 15:35:56.926977 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:56.926935    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ft4rq" podStartSLOduration=34.233351805 podStartE2EDuration="41.926919488s" totalImagesPullingTime="7.693567683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 15:35:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:35:56.926143847 +0000 UTC m=+81.584936506" watchObservedRunningTime="2026-09-18 15:35:56.926919488 +0000 UTC m=+81.585712149"
Sep 18 15:35:57.217583 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:57.217492    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:57.217583 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:57.217550    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:57.222263 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:57.222239    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:57.913931 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:57.913901    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:35:58.337694 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.337654    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"]
Sep 18 15:35:58.342405 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.342387    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.347639 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.347613    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-vvgzv\""
Sep 18 15:35:58.347742 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.347639    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 18 15:35:58.347904 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.347890    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 18 15:35:58.361039 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.361015    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"]
Sep 18 15:35:58.385161 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.385128    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xlpxh"]
Sep 18 15:35:58.388756 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.388738    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.392514 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.392488    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 15:35:58.392514 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.392502    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 15:35:58.393189 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.393023    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 15:35:58.393189 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.393079    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kpxdd\""
Sep 18 15:35:58.481343 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481311    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.481343 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481350    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.481593 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481375    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a9367d11-2046-437f-b049-0f496b2835f0-root\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.481593 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481455    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4cpb\" (UniqueName: \"kubernetes.io/projected/a9367d11-2046-437f-b049-0f496b2835f0-kube-api-access-r4cpb\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.481593 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481501    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9367d11-2046-437f-b049-0f496b2835f0-sys\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.481593 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481527    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9367d11-2046-437f-b049-0f496b2835f0-metrics-client-ca\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.481593 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481554    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bvw6\" (UniqueName: \"kubernetes.io/projected/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-kube-api-access-7bvw6\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.481850 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481602    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-wtmp\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.481850 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481634    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-textfile\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.481850 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481669    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.481850 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481708    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-tls\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.481850 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481731    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.481850 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.481777    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583029 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.582994    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583029 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583031    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.583284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583053    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a9367d11-2046-437f-b049-0f496b2835f0-root\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583080    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4cpb\" (UniqueName: \"kubernetes.io/projected/a9367d11-2046-437f-b049-0f496b2835f0-kube-api-access-r4cpb\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583105    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9367d11-2046-437f-b049-0f496b2835f0-sys\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583126    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9367d11-2046-437f-b049-0f496b2835f0-metrics-client-ca\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583150    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bvw6\" (UniqueName: \"kubernetes.io/projected/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-kube-api-access-7bvw6\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.583284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583176    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9367d11-2046-437f-b049-0f496b2835f0-sys\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583178    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-wtmp\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583284 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583248    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-textfile\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583731 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583289    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-wtmp\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583731 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583290    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.583731 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583347    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-tls\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583731 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583370    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.583731 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583150    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a9367d11-2046-437f-b049-0f496b2835f0-root\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.583731 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583425    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.584054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583854    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9367d11-2046-437f-b049-0f496b2835f0-metrics-client-ca\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.584054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583863    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-textfile\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.584054 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:58.583951    2585 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 18 15:35:58.584054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.583965    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.584054 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:35:58.584015    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-openshift-state-metrics-tls podName:474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d nodeName:}" failed. No retries permitted until 2026-09-18 15:35:59.083996291 +0000 UTC m=+83.742788931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-rgkn6" (UID: "474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d") : secret "openshift-state-metrics-tls" not found
Sep 18 15:35:58.584240 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.584056    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.585703 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.585670    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.586269 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.586239    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-tls\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.586341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.586324    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9367d11-2046-437f-b049-0f496b2835f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.601151 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.601095    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4cpb\" (UniqueName: \"kubernetes.io/projected/a9367d11-2046-437f-b049-0f496b2835f0-kube-api-access-r4cpb\") pod \"node-exporter-xlpxh\" (UID: \"a9367d11-2046-437f-b049-0f496b2835f0\") " pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.601244 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.601232    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bvw6\" (UniqueName: \"kubernetes.io/projected/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-kube-api-access-7bvw6\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:58.699665 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.699626    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xlpxh"
Sep 18 15:35:58.712265 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:58.712239    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9367d11_2046_437f_b049_0f496b2835f0.slice/crio-a9c5b1ebe4d88d0fb1cbdc087ba3f547cb4f2b3b4424cdff5ecebd17b0469981 WatchSource:0}: Error finding container a9c5b1ebe4d88d0fb1cbdc087ba3f547cb4f2b3b4424cdff5ecebd17b0469981: Status 404 returned error can't find the container with id a9c5b1ebe4d88d0fb1cbdc087ba3f547cb4f2b3b4424cdff5ecebd17b0469981
Sep 18 15:35:58.914899 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:58.914807    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlpxh" event={"ID":"a9367d11-2046-437f-b049-0f496b2835f0","Type":"ContainerStarted","Data":"a9c5b1ebe4d88d0fb1cbdc087ba3f547cb4f2b3b4424cdff5ecebd17b0469981"}
Sep 18 15:35:59.088553 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:59.088517    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:59.091159 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:59.091137    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-rgkn6\" (UID: \"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:59.252261 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:59.252230    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"
Sep 18 15:35:59.526816 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:59.526736    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6"]
Sep 18 15:35:59.529993 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:35:59.529971    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474c6af7_f61b_4f0c_b1f5_1fe06ca7cf8d.slice/crio-213d118ec6a111ab6df396a53892804fffc4c35ea9b996159d178d0b821d4876 WatchSource:0}: Error finding container 213d118ec6a111ab6df396a53892804fffc4c35ea9b996159d178d0b821d4876: Status 404 returned error can't find the container with id 213d118ec6a111ab6df396a53892804fffc4c35ea9b996159d178d0b821d4876
Sep 18 15:35:59.921171 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:59.921138    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6" event={"ID":"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d","Type":"ContainerStarted","Data":"4614e1721f58ae56c5830113d7cbc7be78a20c3cd8317881140b66920048a2c0"}
Sep 18 15:35:59.921457 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:59.921256    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6" event={"ID":"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d","Type":"ContainerStarted","Data":"ed1e5b1542a1f5553e81576d65f78236c7de67421459e0dcd490032aaf5af35b"}
Sep 18 15:35:59.921457 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:35:59.921289    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6" event={"ID":"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d","Type":"ContainerStarted","Data":"213d118ec6a111ab6df396a53892804fffc4c35ea9b996159d178d0b821d4876"}
Sep 18 15:36:00.930045 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:00.930013    2585 generic.go:356] "Generic (PLEG): container finished" podID="a9367d11-2046-437f-b049-0f496b2835f0" containerID="3f6798f55e5cdd1afc169b1c0e5421080c4b03424f457bf4d4481c8fce6e7293" exitCode=0
Sep 18 15:36:00.930506 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:00.930092    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlpxh" event={"ID":"a9367d11-2046-437f-b049-0f496b2835f0","Type":"ContainerDied","Data":"3f6798f55e5cdd1afc169b1c0e5421080c4b03424f457bf4d4481c8fce6e7293"}
Sep 18 15:36:01.935726 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:01.935686    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlpxh" event={"ID":"a9367d11-2046-437f-b049-0f496b2835f0","Type":"ContainerStarted","Data":"7b775eef14c93de626e7cfbf7f0af8e5fad06efee45e3bf4f9d240eb54da744a"}
Sep 18 15:36:01.936258 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:01.935743    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlpxh" event={"ID":"a9367d11-2046-437f-b049-0f496b2835f0","Type":"ContainerStarted","Data":"8e361822d08a8c53ea4badf008fb2334f5c0ac45872aaa0393ed5c37728339d9"}
Sep 18 15:36:01.937906 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:01.937879    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6" event={"ID":"474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d","Type":"ContainerStarted","Data":"603b260cc0df8886713ab54d2388a8e695505b69f27ea99b59df94ff3e878b4d"}
Sep 18 15:36:01.958205 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:01.958165    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xlpxh" podStartSLOduration=2.709794466 podStartE2EDuration="3.958151768s" totalImagesPullingTime="1.186568471s" totalInitContainerRuntime="61.788831ms" isStatefulPod=true podCreationTimestamp="2026-09-18 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:36:01.956926475 +0000 UTC m=+86.615719136" watchObservedRunningTime="2026-09-18 15:36:01.958151768 +0000 UTC m=+86.616944427"
Sep 18 15:36:01.975783 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:01.975741    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rgkn6" podStartSLOduration=2.664772086 podStartE2EDuration="3.975728236s" totalImagesPullingTime="1.31095615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:36:01.973872193 +0000 UTC m=+86.632664850" watchObservedRunningTime="2026-09-18 15:36:01.975728236 +0000 UTC m=+86.634520896"
Sep 18 15:36:02.759809 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.759774    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7f4c545679-fn6nc"]
Sep 18 15:36:02.764887 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.764871    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:02.772081 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.772065    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fitcni9oljo20\""
Sep 18 15:36:02.772210 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.772115    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 18 15:36:02.776377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.773429    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 15:36:02.776377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.773508    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rbrjk\""
Sep 18 15:36:02.776377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.773536    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 18 15:36:02.776377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.773812    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 18 15:36:02.781233 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.781207    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f4c545679-fn6nc"]
Sep 18 15:36:02.921179 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.921136    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c776300e-fe76-4bd8-9d60-a4ead7aff655-secret-metrics-server-client-certs\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:02.921359 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.921188    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c776300e-fe76-4bd8-9d60-a4ead7aff655-audit-log\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:02.921359 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.921209    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c776300e-fe76-4bd8-9d60-a4ead7aff655-metrics-server-audit-profiles\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:02.921359 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.921259    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c776300e-fe76-4bd8-9d60-a4ead7aff655-secret-metrics-server-tls\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:02.921359 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.921277    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljklk\" (UniqueName: \"kubernetes.io/projected/c776300e-fe76-4bd8-9d60-a4ead7aff655-kube-api-access-ljklk\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:02.921359 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.921330    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c776300e-fe76-4bd8-9d60-a4ead7aff655-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:02.921359 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:02.921348    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c776300e-fe76-4bd8-9d60-a4ead7aff655-client-ca-bundle\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.021939 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.021849    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c776300e-fe76-4bd8-9d60-a4ead7aff655-audit-log\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.021939 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.021892    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c776300e-fe76-4bd8-9d60-a4ead7aff655-metrics-server-audit-profiles\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.022446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.022258    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c776300e-fe76-4bd8-9d60-a4ead7aff655-audit-log\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.022446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.022265    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c776300e-fe76-4bd8-9d60-a4ead7aff655-secret-metrics-server-tls\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.022446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.022310    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljklk\" (UniqueName: \"kubernetes.io/projected/c776300e-fe76-4bd8-9d60-a4ead7aff655-kube-api-access-ljklk\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.022446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.022404    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c776300e-fe76-4bd8-9d60-a4ead7aff655-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.022446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.022437    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c776300e-fe76-4bd8-9d60-a4ead7aff655-client-ca-bundle\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.022686 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.022487    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c776300e-fe76-4bd8-9d60-a4ead7aff655-secret-metrics-server-client-certs\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.022963 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.022934    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c776300e-fe76-4bd8-9d60-a4ead7aff655-metrics-server-audit-profiles\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.023120 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.023096    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c776300e-fe76-4bd8-9d60-a4ead7aff655-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.024740 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.024720    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c776300e-fe76-4bd8-9d60-a4ead7aff655-secret-metrics-server-tls\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.025167 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.025151    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c776300e-fe76-4bd8-9d60-a4ead7aff655-secret-metrics-server-client-certs\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.025306 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.025286    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c776300e-fe76-4bd8-9d60-a4ead7aff655-client-ca-bundle\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.037106 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.037083    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljklk\" (UniqueName: \"kubernetes.io/projected/c776300e-fe76-4bd8-9d60-a4ead7aff655-kube-api-access-ljklk\") pod \"metrics-server-7f4c545679-fn6nc\" (UID: \"c776300e-fe76-4bd8-9d60-a4ead7aff655\") " pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.047926 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.047905    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"]
Sep 18 15:36:03.053299 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.053286    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:03.059273 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.059254    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 18 15:36:03.059796 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.059737    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cc4sx\""
Sep 18 15:36:03.069000 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.068979    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"]
Sep 18 15:36:03.073331 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.073309    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:03.123271 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.123229    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5zqx\" (UniqueName: \"kubernetes.io/projected/74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38-kube-api-access-h5zqx\") pod \"monitoring-plugin-5984459ff8-nw59m\" (UID: \"74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:03.123429 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.123310    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-nw59m\" (UID: \"74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:03.224652 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.224618    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5zqx\" (UniqueName: \"kubernetes.io/projected/74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38-kube-api-access-h5zqx\") pod \"monitoring-plugin-5984459ff8-nw59m\" (UID: \"74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:03.224920 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.224668    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-nw59m\" (UID: \"74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:03.224920 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:36:03.224780    2585 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 18 15:36:03.224920 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:36:03.224832    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38-monitoring-plugin-cert podName:74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38 nodeName:}" failed. No retries permitted until 2026-09-18 15:36:03.724816933 +0000 UTC m=+88.383609571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-nw59m" (UID: "74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38") : secret "monitoring-plugin-cert" not found
Sep 18 15:36:03.239303 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.239272    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5zqx\" (UniqueName: \"kubernetes.io/projected/74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38-kube-api-access-h5zqx\") pod \"monitoring-plugin-5984459ff8-nw59m\" (UID: \"74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:03.364158 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.364091    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f4c545679-fn6nc"]
Sep 18 15:36:03.366825 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:36:03.366799    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc776300e_fe76_4bd8_9d60_a4ead7aff655.slice/crio-b38b51a86b1041e288d3683921e43187020384e9d9880644ebd0bab3eb421a5b WatchSource:0}: Error finding container b38b51a86b1041e288d3683921e43187020384e9d9880644ebd0bab3eb421a5b: Status 404 returned error can't find the container with id b38b51a86b1041e288d3683921e43187020384e9d9880644ebd0bab3eb421a5b
Sep 18 15:36:03.729595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.729513    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-nw59m\" (UID: \"74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:03.731804 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.731780    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-nw59m\" (UID: \"74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:03.946730 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.946697    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc" event={"ID":"c776300e-fe76-4bd8-9d60-a4ead7aff655","Type":"ContainerStarted","Data":"b38b51a86b1041e288d3683921e43187020384e9d9880644ebd0bab3eb421a5b"}
Sep 18 15:36:03.961968 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:03.961942    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:04.248336 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:04.248303    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"]
Sep 18 15:36:04.251629 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:36:04.251600    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b7e6bb_9cbc_4cca_aef5_f0f9721cfa38.slice/crio-eab3d46021460746dcbc4576464fab7e2176eb759edcc1ffac7303626737b118 WatchSource:0}: Error finding container eab3d46021460746dcbc4576464fab7e2176eb759edcc1ffac7303626737b118: Status 404 returned error can't find the container with id eab3d46021460746dcbc4576464fab7e2176eb759edcc1ffac7303626737b118
Sep 18 15:36:04.953034 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:04.952997    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc" event={"ID":"c776300e-fe76-4bd8-9d60-a4ead7aff655","Type":"ContainerStarted","Data":"d21708915e5280aed7fa2f15263e76e330adc3d89230b0d4d125d959a91ecea2"}
Sep 18 15:36:04.954675 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:04.954613    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m" event={"ID":"74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38","Type":"ContainerStarted","Data":"eab3d46021460746dcbc4576464fab7e2176eb759edcc1ffac7303626737b118"}
Sep 18 15:36:05.651214 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:05.651142    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6494946bcd-bttdp" podUID="dc407841-e310-4671-bd17-fdc0fdd5d38a" containerName="registry" containerID="cri-o://4d2e468ad7e6a78a0e7ec217f007d104541f079bc5c3f36626b0f555f43711ee" gracePeriod=30
Sep 18 15:36:05.959485 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:05.959446    2585 generic.go:356] "Generic (PLEG): container finished" podID="dc407841-e310-4671-bd17-fdc0fdd5d38a" containerID="4d2e468ad7e6a78a0e7ec217f007d104541f079bc5c3f36626b0f555f43711ee" exitCode=0
Sep 18 15:36:05.959584 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:05.959514    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6494946bcd-bttdp" event={"ID":"dc407841-e310-4671-bd17-fdc0fdd5d38a","Type":"ContainerDied","Data":"4d2e468ad7e6a78a0e7ec217f007d104541f079bc5c3f36626b0f555f43711ee"}
Sep 18 15:36:05.980010 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:05.979983    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:36:06.003726 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.003669    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc" podStartSLOduration=2.6160238099999997 podStartE2EDuration="4.003653148s" totalImagesPullingTime="1.387629338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:36:05.005140687 +0000 UTC m=+89.663933348" watchObservedRunningTime="2026-09-18 15:36:06.003653148 +0000 UTC m=+90.662445807"
Sep 18 15:36:06.153757 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.153713    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-bound-sa-token\") pod \"dc407841-e310-4671-bd17-fdc0fdd5d38a\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") "
Sep 18 15:36:06.153757 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.153763    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-trusted-ca\") pod \"dc407841-e310-4671-bd17-fdc0fdd5d38a\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") "
Sep 18 15:36:06.154001 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.153828    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-installation-pull-secrets\") pod \"dc407841-e310-4671-bd17-fdc0fdd5d38a\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") "
Sep 18 15:36:06.154001 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.153876    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") pod \"dc407841-e310-4671-bd17-fdc0fdd5d38a\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") "
Sep 18 15:36:06.154001 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.153911    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc407841-e310-4671-bd17-fdc0fdd5d38a-ca-trust-extracted\") pod \"dc407841-e310-4671-bd17-fdc0fdd5d38a\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") "
Sep 18 15:36:06.154001 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.153965    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-image-registry-private-configuration\") pod \"dc407841-e310-4671-bd17-fdc0fdd5d38a\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") "
Sep 18 15:36:06.154201 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.154008    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6rbn\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-kube-api-access-j6rbn\") pod \"dc407841-e310-4671-bd17-fdc0fdd5d38a\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") "
Sep 18 15:36:06.154201 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.154048    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-certificates\") pod \"dc407841-e310-4671-bd17-fdc0fdd5d38a\" (UID: \"dc407841-e310-4671-bd17-fdc0fdd5d38a\") "
Sep 18 15:36:06.154308 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.154279    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "dc407841-e310-4671-bd17-fdc0fdd5d38a" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:36:06.155506 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.154560    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "dc407841-e310-4671-bd17-fdc0fdd5d38a" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:36:06.156377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.156351    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "dc407841-e310-4671-bd17-fdc0fdd5d38a" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:36:06.156480 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.156433    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "dc407841-e310-4671-bd17-fdc0fdd5d38a" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:36:06.156581 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.156557    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "dc407841-e310-4671-bd17-fdc0fdd5d38a" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:36:06.156640 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.156600    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "dc407841-e310-4671-bd17-fdc0fdd5d38a" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:36:06.156691 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.156651    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-kube-api-access-j6rbn" (OuterVolumeSpecName: "kube-api-access-j6rbn") pod "dc407841-e310-4671-bd17-fdc0fdd5d38a" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a"). InnerVolumeSpecName "kube-api-access-j6rbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:36:06.162522 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.162499    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc407841-e310-4671-bd17-fdc0fdd5d38a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "dc407841-e310-4671-bd17-fdc0fdd5d38a" (UID: "dc407841-e310-4671-bd17-fdc0fdd5d38a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:36:06.255595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.255548    2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-installation-pull-secrets\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:06.255595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.255595    2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:06.255777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.255606    2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc407841-e310-4671-bd17-fdc0fdd5d38a-ca-trust-extracted\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:06.255777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.255617    2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc407841-e310-4671-bd17-fdc0fdd5d38a-image-registry-private-configuration\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:06.255777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.255626    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6rbn\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-kube-api-access-j6rbn\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:06.255777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.255635    2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-registry-certificates\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:06.255777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.255643    2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc407841-e310-4671-bd17-fdc0fdd5d38a-bound-sa-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:06.255777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.255651    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc407841-e310-4671-bd17-fdc0fdd5d38a-trusted-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:06.966350 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.966306    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m" event={"ID":"74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38","Type":"ContainerStarted","Data":"2321bea702d880e5660acdfffa81b596e1d06068642046a399502124961d473d"}
Sep 18 15:36:06.966780 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.966453    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:06.967681 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.967658    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6494946bcd-bttdp" event={"ID":"dc407841-e310-4671-bd17-fdc0fdd5d38a","Type":"ContainerDied","Data":"0a4e26c2adfa9354a06109e91c161d8137fe41a8ce24c8c32bbe08529271f145"}
Sep 18 15:36:06.967787 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.967682    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6494946bcd-bttdp"
Sep 18 15:36:06.967787 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.967699    2585 scope.go:117] "RemoveContainer" containerID="4d2e468ad7e6a78a0e7ec217f007d104541f079bc5c3f36626b0f555f43711ee"
Sep 18 15:36:06.971781 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.971760    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m"
Sep 18 15:36:06.982513 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.982468    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-nw59m" podStartSLOduration=2.350813655 podStartE2EDuration="3.982454686s" totalImagesPullingTime="1.631641031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:36:06.981508426 +0000 UTC m=+91.640301086" watchObservedRunningTime="2026-09-18 15:36:06.982454686 +0000 UTC m=+91.641247379"
Sep 18 15:36:06.994291 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.994265    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6494946bcd-bttdp"]
Sep 18 15:36:06.998226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:06.998205    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6494946bcd-bttdp"]
Sep 18 15:36:07.521601 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:07.521574    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc407841-e310-4671-bd17-fdc0fdd5d38a" path="/var/lib/kubelet/pods/dc407841-e310-4671-bd17-fdc0fdd5d38a/volumes"
Sep 18 15:36:14.009064 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:14.009030    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-594f8d6749-gglqm"]
Sep 18 15:36:23.074454 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:23.074420    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:23.074876 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:23.074490    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:26.907663 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:26.907633    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qq2zh"
Sep 18 15:36:39.032563 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.032502    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-594f8d6749-gglqm" podUID="70120c62-b291-40ff-83ef-eb60ea61f8dc" containerName="console" containerID="cri-o://ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b" gracePeriod=15
Sep 18 15:36:39.302831 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.302762    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:36:39.396126 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.396099    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-oauth-config\") pod \"70120c62-b291-40ff-83ef-eb60ea61f8dc\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") "
Sep 18 15:36:39.396300 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.396184    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-oauth-serving-cert\") pod \"70120c62-b291-40ff-83ef-eb60ea61f8dc\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") "
Sep 18 15:36:39.396300 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.396219    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wb6qb\" (UniqueName: \"kubernetes.io/projected/70120c62-b291-40ff-83ef-eb60ea61f8dc-kube-api-access-wb6qb\") pod \"70120c62-b291-40ff-83ef-eb60ea61f8dc\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") "
Sep 18 15:36:39.396300 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.396242    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-config\") pod \"70120c62-b291-40ff-83ef-eb60ea61f8dc\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") "
Sep 18 15:36:39.396300 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.396257    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-serving-cert\") pod \"70120c62-b291-40ff-83ef-eb60ea61f8dc\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") "
Sep 18 15:36:39.396300 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.396274    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-service-ca\") pod \"70120c62-b291-40ff-83ef-eb60ea61f8dc\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") "
Sep 18 15:36:39.396300 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.396297    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-trusted-ca-bundle\") pod \"70120c62-b291-40ff-83ef-eb60ea61f8dc\" (UID: \"70120c62-b291-40ff-83ef-eb60ea61f8dc\") "
Sep 18 15:36:39.397460 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.397145    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-config" (OuterVolumeSpecName: "console-config") pod "70120c62-b291-40ff-83ef-eb60ea61f8dc" (UID: "70120c62-b291-40ff-83ef-eb60ea61f8dc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:36:39.397460 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.397159    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "70120c62-b291-40ff-83ef-eb60ea61f8dc" (UID: "70120c62-b291-40ff-83ef-eb60ea61f8dc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:36:39.397460 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.397226    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-service-ca" (OuterVolumeSpecName: "service-ca") pod "70120c62-b291-40ff-83ef-eb60ea61f8dc" (UID: "70120c62-b291-40ff-83ef-eb60ea61f8dc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:36:39.397460 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.397439    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "70120c62-b291-40ff-83ef-eb60ea61f8dc" (UID: "70120c62-b291-40ff-83ef-eb60ea61f8dc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:36:39.402811 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.402621    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "70120c62-b291-40ff-83ef-eb60ea61f8dc" (UID: "70120c62-b291-40ff-83ef-eb60ea61f8dc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:36:39.403322 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.403300    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "70120c62-b291-40ff-83ef-eb60ea61f8dc" (UID: "70120c62-b291-40ff-83ef-eb60ea61f8dc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:36:39.403406 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.403310    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70120c62-b291-40ff-83ef-eb60ea61f8dc-kube-api-access-wb6qb" (OuterVolumeSpecName: "kube-api-access-wb6qb") pod "70120c62-b291-40ff-83ef-eb60ea61f8dc" (UID: "70120c62-b291-40ff-83ef-eb60ea61f8dc"). InnerVolumeSpecName "kube-api-access-wb6qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:36:39.497729 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.497687    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb6qb\" (UniqueName: \"kubernetes.io/projected/70120c62-b291-40ff-83ef-eb60ea61f8dc-kube-api-access-wb6qb\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:39.497894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.497758    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-config\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:39.497894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.497774    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-serving-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:39.497894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.497787    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:39.497894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.497800    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-trusted-ca-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:39.497894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.497813    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70120c62-b291-40ff-83ef-eb60ea61f8dc-console-oauth-config\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:39.497894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:39.497825    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70120c62-b291-40ff-83ef-eb60ea61f8dc-oauth-serving-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:36:40.095732 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:40.095697    2585 generic.go:356] "Generic (PLEG): container finished" podID="70120c62-b291-40ff-83ef-eb60ea61f8dc" containerID="ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b" exitCode=0
Sep 18 15:36:40.096180 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:40.095754    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f8d6749-gglqm"
Sep 18 15:36:40.096180 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:40.095779    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f8d6749-gglqm" event={"ID":"70120c62-b291-40ff-83ef-eb60ea61f8dc","Type":"ContainerDied","Data":"ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b"}
Sep 18 15:36:40.096180 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:40.095867    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f8d6749-gglqm" event={"ID":"70120c62-b291-40ff-83ef-eb60ea61f8dc","Type":"ContainerDied","Data":"7b89586677d44f2b47e61f15dff139a6481ff2fc3fc104500cdc183c9366e324"}
Sep 18 15:36:40.096180 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:40.095888    2585 scope.go:117] "RemoveContainer" containerID="ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b"
Sep 18 15:36:40.103720 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:40.103702    2585 scope.go:117] "RemoveContainer" containerID="ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b"
Sep 18 15:36:40.104053 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:36:40.104030    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b\": container with ID starting with ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b not found: ID does not exist" containerID="ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b"
Sep 18 15:36:40.104112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:40.104066    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b"} err="failed to get container status \"ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b\": rpc error: code = NotFound desc = could not find container \"ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b\": container with ID starting with ad464bd060ae0973aa846a0876bec2520906e5c624c3d4c501646f323c7f8f3b not found: ID does not exist"
Sep 18 15:36:40.111590 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:40.111568    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-594f8d6749-gglqm"]
Sep 18 15:36:40.115007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:40.114987    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-594f8d6749-gglqm"]
Sep 18 15:36:41.521215 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:41.521186    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70120c62-b291-40ff-83ef-eb60ea61f8dc" path="/var/lib/kubelet/pods/70120c62-b291-40ff-83ef-eb60ea61f8dc/volumes"
Sep 18 15:36:43.079045 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:43.079020    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:36:43.082905 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:36:43.082885    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7f4c545679-fn6nc"
Sep 18 15:37:25.394197 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.394122    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54f48c88c7-2zswk"]
Sep 18 15:37:25.394588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.394394    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc407841-e310-4671-bd17-fdc0fdd5d38a" containerName="registry"
Sep 18 15:37:25.394588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.394404    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc407841-e310-4671-bd17-fdc0fdd5d38a" containerName="registry"
Sep 18 15:37:25.394588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.394418    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70120c62-b291-40ff-83ef-eb60ea61f8dc" containerName="console"
Sep 18 15:37:25.394588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.394423    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="70120c62-b291-40ff-83ef-eb60ea61f8dc" containerName="console"
Sep 18 15:37:25.394588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.394471    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="70120c62-b291-40ff-83ef-eb60ea61f8dc" containerName="console"
Sep 18 15:37:25.394588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.394477    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc407841-e310-4671-bd17-fdc0fdd5d38a" containerName="registry"
Sep 18 15:37:25.397159 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.397144    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.400521 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.400501    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 15:37:25.400785 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.400772    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 15:37:25.401941 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.401918    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 15:37:25.402054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.401943    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8fq62\""
Sep 18 15:37:25.402054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.401969    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 15:37:25.402054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.402039    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 15:37:25.402248 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.402235    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 15:37:25.402287 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.402265    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 15:37:25.406573 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.406551    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 15:37:25.415631 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.415611    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f48c88c7-2zswk"]
Sep 18 15:37:25.440529 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.440507    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-serving-cert\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.440632 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.440539    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-service-ca\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.440632 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.440574    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-oauth-config\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.440702 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.440628    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp6nf\" (UniqueName: \"kubernetes.io/projected/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-kube-api-access-bp6nf\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.440702 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.440657    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-config\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.440702 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.440678    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-oauth-serving-cert\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.440702 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.440694    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-trusted-ca-bundle\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.541178 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.541152    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-oauth-config\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.541178 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.541183    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp6nf\" (UniqueName: \"kubernetes.io/projected/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-kube-api-access-bp6nf\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.541389 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.541205    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-config\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.541389 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.541225    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-oauth-serving-cert\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.541389 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.541241    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-trusted-ca-bundle\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.541389 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.541275    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-serving-cert\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.541389 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.541307    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-service-ca\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.542040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.542014    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-config\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.542169 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.542091    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-trusted-ca-bundle\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.542169 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.542105    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-service-ca\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.542169 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.542094    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-oauth-serving-cert\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.543753 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.543731    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-oauth-config\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.543858 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.543776    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-serving-cert\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.549912 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.549890    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp6nf\" (UniqueName: \"kubernetes.io/projected/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-kube-api-access-bp6nf\") pod \"console-54f48c88c7-2zswk\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") " pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.707114 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.707027    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:25.987893 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:25.987860    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f48c88c7-2zswk"]
Sep 18 15:37:25.991283 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:37:25.991246    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115bf4d7_af5a_42c3_8d14_ab26ac66f4ec.slice/crio-e2db5a76e9a0d6e2e0bab4e674d7e59ef9ba9c5c5ab5630df9ec131b78441c54 WatchSource:0}: Error finding container e2db5a76e9a0d6e2e0bab4e674d7e59ef9ba9c5c5ab5630df9ec131b78441c54: Status 404 returned error can't find the container with id e2db5a76e9a0d6e2e0bab4e674d7e59ef9ba9c5c5ab5630df9ec131b78441c54
Sep 18 15:37:26.257040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:26.256949    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f48c88c7-2zswk" event={"ID":"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec","Type":"ContainerStarted","Data":"976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3"}
Sep 18 15:37:26.257040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:26.257008    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f48c88c7-2zswk" event={"ID":"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec","Type":"ContainerStarted","Data":"e2db5a76e9a0d6e2e0bab4e674d7e59ef9ba9c5c5ab5630df9ec131b78441c54"}
Sep 18 15:37:26.275161 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:26.275113    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54f48c88c7-2zswk" podStartSLOduration=1.2750972090000001 podStartE2EDuration="1.275097209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:37:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:37:26.273720721 +0000 UTC m=+170.932513407" watchObservedRunningTime="2026-09-18 15:37:26.275097209 +0000 UTC m=+170.933889869"
Sep 18 15:37:33.402595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:33.402558    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f48c88c7-2zswk"]
Sep 18 15:37:35.707917 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:35.707885    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:58.420954 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.420897    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54f48c88c7-2zswk" podUID="115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" containerName="console" containerID="cri-o://976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3" gracePeriod=15
Sep 18 15:37:58.658252 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.658229    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:58.785293 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785250    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-oauth-serving-cert\") pod \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") "
Sep 18 15:37:58.785454 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785323    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-config\") pod \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") "
Sep 18 15:37:58.785495 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785474    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-service-ca\") pod \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") "
Sep 18 15:37:58.785534 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785515    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-oauth-config\") pod \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") "
Sep 18 15:37:58.785567 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785544    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-serving-cert\") pod \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") "
Sep 18 15:37:58.785600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785588    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp6nf\" (UniqueName: \"kubernetes.io/projected/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-kube-api-access-bp6nf\") pod \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") "
Sep 18 15:37:58.785632 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785607    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" (UID: "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:37:58.785674 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785632    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-trusted-ca-bundle\") pod \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\" (UID: \"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec\") "
Sep 18 15:37:58.785674 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785665    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-config" (OuterVolumeSpecName: "console-config") pod "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" (UID: "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:37:58.785854 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785819    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-oauth-serving-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:37:58.785854 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785854    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-config\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:37:58.786037 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.785883    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-service-ca" (OuterVolumeSpecName: "service-ca") pod "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" (UID: "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:37:58.786109 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.786085    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" (UID: "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:37:58.787570 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.787549    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" (UID: "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:37:58.787663 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.787570    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" (UID: "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:37:58.787663 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.787640    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-kube-api-access-bp6nf" (OuterVolumeSpecName: "kube-api-access-bp6nf") pod "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" (UID: "115bf4d7-af5a-42c3-8d14-ab26ac66f4ec"). InnerVolumeSpecName "kube-api-access-bp6nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:37:58.886990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.886951    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp6nf\" (UniqueName: \"kubernetes.io/projected/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-kube-api-access-bp6nf\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:37:58.886990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.886983    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-trusted-ca-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:37:58.886990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.886994    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:37:58.886990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.887002    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-oauth-config\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:37:58.887245 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:58.887010    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec-console-serving-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:37:59.371481 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.371447    2585 generic.go:356] "Generic (PLEG): container finished" podID="115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" containerID="976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3" exitCode=0
Sep 18 15:37:59.371640 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.371490    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f48c88c7-2zswk" event={"ID":"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec","Type":"ContainerDied","Data":"976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3"}
Sep 18 15:37:59.371640 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.371496    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f48c88c7-2zswk"
Sep 18 15:37:59.371640 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.371530    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f48c88c7-2zswk" event={"ID":"115bf4d7-af5a-42c3-8d14-ab26ac66f4ec","Type":"ContainerDied","Data":"e2db5a76e9a0d6e2e0bab4e674d7e59ef9ba9c5c5ab5630df9ec131b78441c54"}
Sep 18 15:37:59.371640 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.371548    2585 scope.go:117] "RemoveContainer" containerID="976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3"
Sep 18 15:37:59.379562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.379542    2585 scope.go:117] "RemoveContainer" containerID="976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3"
Sep 18 15:37:59.379825 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:37:59.379806    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3\": container with ID starting with 976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3 not found: ID does not exist" containerID="976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3"
Sep 18 15:37:59.379888 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.379849    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3"} err="failed to get container status \"976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3\": rpc error: code = NotFound desc = could not find container \"976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3\": container with ID starting with 976fabf73fb0a6b673dbb09e877290faac890ab243df81df956dc2087a3f47b3 not found: ID does not exist"
Sep 18 15:37:59.391073 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.391048    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f48c88c7-2zswk"]
Sep 18 15:37:59.395163 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.395144    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54f48c88c7-2zswk"]
Sep 18 15:37:59.521597 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:37:59.521569    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" path="/var/lib/kubelet/pods/115bf4d7-af5a-42c3-8d14-ab26ac66f4ec/volumes"
Sep 18 15:39:37.353275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:37.353244    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:39:37.354989 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:37.354973    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:39:37.360120 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:37.360106    2585 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 15:39:53.721911 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.721884    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"]
Sep 18 15:39:53.724226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.722166    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" containerName="console"
Sep 18 15:39:53.724226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.722181    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" containerName="console"
Sep 18 15:39:53.724226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.722228    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="115bf4d7-af5a-42c3-8d14-ab26ac66f4ec" containerName="console"
Sep 18 15:39:53.725107 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.725093    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:53.728050 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.728027    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g427q\""
Sep 18 15:39:53.728159 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.728143    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 15:39:53.729129 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.729115    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 15:39:53.742931 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.742908    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"]
Sep 18 15:39:53.797803 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.797772    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:53.797986 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.797822    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:53.797986 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.797861    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdwk8\" (UniqueName: \"kubernetes.io/projected/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-kube-api-access-hdwk8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:53.898655 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.898623    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:53.898807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.898668    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:53.898807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.898694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdwk8\" (UniqueName: \"kubernetes.io/projected/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-kube-api-access-hdwk8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:53.899019 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.898998    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:53.899092 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.899059    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:53.907545 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:53.907524    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdwk8\" (UniqueName: \"kubernetes.io/projected/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-kube-api-access-hdwk8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:54.033637 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:54.033600    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:39:54.304158 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:54.304082    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"]
Sep 18 15:39:54.306999 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:39:54.306970    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc375a2f3_fd8d_4319_bb29_9afdfbce53d4.slice/crio-8b8eeebd834a98b92c1d9a385d17ffb86e53e9524e0c06d667cbb05c111f4fa7 WatchSource:0}: Error finding container 8b8eeebd834a98b92c1d9a385d17ffb86e53e9524e0c06d667cbb05c111f4fa7: Status 404 returned error can't find the container with id 8b8eeebd834a98b92c1d9a385d17ffb86e53e9524e0c06d667cbb05c111f4fa7
Sep 18 15:39:54.308940 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:54.308913    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 15:39:54.752542 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:54.752509    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2" event={"ID":"c375a2f3-fd8d-4319-bb29-9afdfbce53d4","Type":"ContainerStarted","Data":"8b8eeebd834a98b92c1d9a385d17ffb86e53e9524e0c06d667cbb05c111f4fa7"}
Sep 18 15:39:59.773634 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:59.773600    2585 generic.go:356] "Generic (PLEG): container finished" podID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerID="b5c445665d331d4e5303d5831fd446e4236b9fbfb21599669e3ddba34c75ee3b" exitCode=0
Sep 18 15:39:59.773980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:39:59.773675    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2" event={"ID":"c375a2f3-fd8d-4319-bb29-9afdfbce53d4","Type":"ContainerDied","Data":"b5c445665d331d4e5303d5831fd446e4236b9fbfb21599669e3ddba34c75ee3b"}
Sep 18 15:40:02.786030 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:02.785996    2585 generic.go:356] "Generic (PLEG): container finished" podID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerID="ee9c958b4f499300ecc3efe1f6b93fa1da282132b70a3da8f982a2b90079cfbb" exitCode=0
Sep 18 15:40:02.786030 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:02.786036    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2" event={"ID":"c375a2f3-fd8d-4319-bb29-9afdfbce53d4","Type":"ContainerDied","Data":"ee9c958b4f499300ecc3efe1f6b93fa1da282132b70a3da8f982a2b90079cfbb"}
Sep 18 15:40:08.813273 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:08.813234    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2" event={"ID":"c375a2f3-fd8d-4319-bb29-9afdfbce53d4","Type":"ContainerStarted","Data":"0a8cf30b450544342cab01f6f87f389197b27724afa1ddd7a070effa51045520"}
Sep 18 15:40:08.834054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:08.834002    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2" podStartSLOduration=2.539962371 podStartE2EDuration="15.833988258s" totalImagesPullingTime="13.294025887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:39:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:40:08.833313102 +0000 UTC m=+333.492105759" watchObservedRunningTime="2026-09-18 15:40:08.833988258 +0000 UTC m=+333.492780917"
Sep 18 15:40:09.819039 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:09.819003    2585 generic.go:356] "Generic (PLEG): container finished" podID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerID="0a8cf30b450544342cab01f6f87f389197b27724afa1ddd7a070effa51045520" exitCode=0
Sep 18 15:40:09.819421 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:09.819054    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2" event={"ID":"c375a2f3-fd8d-4319-bb29-9afdfbce53d4","Type":"ContainerDied","Data":"0a8cf30b450544342cab01f6f87f389197b27724afa1ddd7a070effa51045520"}
Sep 18 15:40:10.945215 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:10.945194    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:40:11.043624 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.043584    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdwk8\" (UniqueName: \"kubernetes.io/projected/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-kube-api-access-hdwk8\") pod \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") "
Sep 18 15:40:11.043791 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.043645    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-bundle\") pod \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") "
Sep 18 15:40:11.043791 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.043713    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-util\") pod \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\" (UID: \"c375a2f3-fd8d-4319-bb29-9afdfbce53d4\") "
Sep 18 15:40:11.044313 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.044286    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-bundle" (OuterVolumeSpecName: "bundle") pod "c375a2f3-fd8d-4319-bb29-9afdfbce53d4" (UID: "c375a2f3-fd8d-4319-bb29-9afdfbce53d4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:40:11.045814 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.045793    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-kube-api-access-hdwk8" (OuterVolumeSpecName: "kube-api-access-hdwk8") pod "c375a2f3-fd8d-4319-bb29-9afdfbce53d4" (UID: "c375a2f3-fd8d-4319-bb29-9afdfbce53d4"). InnerVolumeSpecName "kube-api-access-hdwk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:40:11.048678 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.048652    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-util" (OuterVolumeSpecName: "util") pod "c375a2f3-fd8d-4319-bb29-9afdfbce53d4" (UID: "c375a2f3-fd8d-4319-bb29-9afdfbce53d4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:40:11.145230 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.145156    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdwk8\" (UniqueName: \"kubernetes.io/projected/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-kube-api-access-hdwk8\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:40:11.145230 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.145182    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:40:11.145230 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.145191    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c375a2f3-fd8d-4319-bb29-9afdfbce53d4-util\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:40:11.827749 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.827668    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2" event={"ID":"c375a2f3-fd8d-4319-bb29-9afdfbce53d4","Type":"ContainerDied","Data":"8b8eeebd834a98b92c1d9a385d17ffb86e53e9524e0c06d667cbb05c111f4fa7"}
Sep 18 15:40:11.827749 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.827699    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754049xn2"
Sep 18 15:40:11.827749 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:11.827717    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b8eeebd834a98b92c1d9a385d17ffb86e53e9524e0c06d667cbb05c111f4fa7"
Sep 18 15:40:16.808598 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.808566    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"]
Sep 18 15:40:16.808994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.808872    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerName="pull"
Sep 18 15:40:16.808994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.808885    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerName="pull"
Sep 18 15:40:16.808994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.808907    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerName="util"
Sep 18 15:40:16.808994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.808912    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerName="util"
Sep 18 15:40:16.808994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.808918    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerName="extract"
Sep 18 15:40:16.808994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.808924    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerName="extract"
Sep 18 15:40:16.808994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.808967    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c375a2f3-fd8d-4319-bb29-9afdfbce53d4" containerName="extract"
Sep 18 15:40:16.853435 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.853395    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"]
Sep 18 15:40:16.853592 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.853518    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"
Sep 18 15:40:16.856309 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.856284    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 18 15:40:16.856454 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.856390    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ncxbk\""
Sep 18 15:40:16.856693 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.856679    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 18 15:40:16.984154 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.984121    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6phxg\" (UID: \"4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"
Sep 18 15:40:16.984154 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:16.984156    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-757rr\" (UniqueName: \"kubernetes.io/projected/4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36-kube-api-access-757rr\") pod \"cert-manager-operator-controller-manager-55c444bf86-6phxg\" (UID: \"4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"
Sep 18 15:40:17.085100 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:17.085006    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6phxg\" (UID: \"4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"
Sep 18 15:40:17.085100 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:17.085038    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-757rr\" (UniqueName: \"kubernetes.io/projected/4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36-kube-api-access-757rr\") pod \"cert-manager-operator-controller-manager-55c444bf86-6phxg\" (UID: \"4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"
Sep 18 15:40:17.085400 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:17.085383    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6phxg\" (UID: \"4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"
Sep 18 15:40:17.099252 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:17.099226    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-757rr\" (UniqueName: \"kubernetes.io/projected/4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36-kube-api-access-757rr\") pod \"cert-manager-operator-controller-manager-55c444bf86-6phxg\" (UID: \"4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"
Sep 18 15:40:17.162926 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:17.162895    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"
Sep 18 15:40:17.462558 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:17.462529    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg"]
Sep 18 15:40:17.463869 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:40:17.463828    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc7594b_5a3c_4f3e_9fe0_38063c0a7d36.slice/crio-f2fd768a639eb237d64a01342fabb77393f802982877ec63ec50c941e167bc32 WatchSource:0}: Error finding container f2fd768a639eb237d64a01342fabb77393f802982877ec63ec50c941e167bc32: Status 404 returned error can't find the container with id f2fd768a639eb237d64a01342fabb77393f802982877ec63ec50c941e167bc32
Sep 18 15:40:17.851982 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:17.851947    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg" event={"ID":"4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36","Type":"ContainerStarted","Data":"f2fd768a639eb237d64a01342fabb77393f802982877ec63ec50c941e167bc32"}
Sep 18 15:40:19.861621 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:19.861534    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg" event={"ID":"4dc7594b-5a3c-4f3e-9fe0-38063c0a7d36","Type":"ContainerStarted","Data":"5916ed1c1476f08e5a995cb48d4b32141e03a7519347d1b30ca746af56947d9c"}
Sep 18 15:40:19.883851 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:19.883735    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6phxg" podStartSLOduration=1.7804914950000001 podStartE2EDuration="3.883715893s" totalImagesPullingTime="2.103224398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:40:19.882198054 +0000 UTC m=+344.540990716" watchObservedRunningTime="2026-09-18 15:40:19.883715893 +0000 UTC m=+344.542508554"
Sep 18 15:40:20.557733 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.557698    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"]
Sep 18 15:40:20.575018 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.574994    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.577763 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.577725    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g427q\""
Sep 18 15:40:20.578053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.578028    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 15:40:20.579722 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.579703    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 15:40:20.580793 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.580770    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"]
Sep 18 15:40:20.715798 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.715766    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h8rd\" (UniqueName: \"kubernetes.io/projected/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-kube-api-access-4h8rd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.715998 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.715806    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.715998 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.715920    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.816780 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.816699    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.816780 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.816761    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.816985 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.816792    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h8rd\" (UniqueName: \"kubernetes.io/projected/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-kube-api-access-4h8rd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.817194 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.817171    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.817232 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.817169    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.828650 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.828622    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h8rd\" (UniqueName: \"kubernetes.io/projected/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-kube-api-access-4h8rd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:20.885912 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:20.885886    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:21.183779 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:21.183756    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"]
Sep 18 15:40:21.185847 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:40:21.185814    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138a6cfd_4449_4ba2_8b09_c4ca1a9a6c52.slice/crio-28c0b73ef6d431d8f2ee3777bcf44a1e53946b8f2c02970607af1a01b03b06fa WatchSource:0}: Error finding container 28c0b73ef6d431d8f2ee3777bcf44a1e53946b8f2c02970607af1a01b03b06fa: Status 404 returned error can't find the container with id 28c0b73ef6d431d8f2ee3777bcf44a1e53946b8f2c02970607af1a01b03b06fa
Sep 18 15:40:21.870516 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:21.870482    2585 generic.go:356] "Generic (PLEG): container finished" podID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerID="f238f08771c6e157d94e9d045cfb42e3d2fbb433952c8a852a9efd6473875355" exitCode=0
Sep 18 15:40:21.870673 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:21.870516    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj" event={"ID":"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52","Type":"ContainerDied","Data":"f238f08771c6e157d94e9d045cfb42e3d2fbb433952c8a852a9efd6473875355"}
Sep 18 15:40:21.870673 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:21.870568    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj" event={"ID":"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52","Type":"ContainerStarted","Data":"28c0b73ef6d431d8f2ee3777bcf44a1e53946b8f2c02970607af1a01b03b06fa"}
Sep 18 15:40:24.885517 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:24.885487    2585 generic.go:356] "Generic (PLEG): container finished" podID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerID="387c5a25c8808b584e818b6f7c59f5fcd6afa962d16e6cb055854d60c183776e" exitCode=0
Sep 18 15:40:24.885909 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:24.885571    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj" event={"ID":"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52","Type":"ContainerDied","Data":"387c5a25c8808b584e818b6f7c59f5fcd6afa962d16e6cb055854d60c183776e"}
Sep 18 15:40:25.891811 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:25.891778    2585 generic.go:356] "Generic (PLEG): container finished" podID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerID="c8eaf40af5e767f5b5a6f256097d20617dc25d7765fdeba08dc6fad033a9a288" exitCode=0
Sep 18 15:40:25.892280 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:25.891858    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj" event={"ID":"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52","Type":"ContainerDied","Data":"c8eaf40af5e767f5b5a6f256097d20617dc25d7765fdeba08dc6fad033a9a288"}
Sep 18 15:40:27.012594 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.012570    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:27.065962 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.065928    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h8rd\" (UniqueName: \"kubernetes.io/projected/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-kube-api-access-4h8rd\") pod \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") "
Sep 18 15:40:27.066145 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.065984    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-bundle\") pod \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") "
Sep 18 15:40:27.066145 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.066028    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-util\") pod \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\" (UID: \"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52\") "
Sep 18 15:40:27.066412 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.066389    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-bundle" (OuterVolumeSpecName: "bundle") pod "138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" (UID: "138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:40:27.068121 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.068095    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-kube-api-access-4h8rd" (OuterVolumeSpecName: "kube-api-access-4h8rd") pod "138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" (UID: "138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52"). InnerVolumeSpecName "kube-api-access-4h8rd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:40:27.070493 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.070472    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-util" (OuterVolumeSpecName: "util") pod "138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" (UID: "138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:40:27.167105 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.167020    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-util\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:40:27.167105 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.167049    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h8rd\" (UniqueName: \"kubernetes.io/projected/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-kube-api-access-4h8rd\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:40:27.167105 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.167059    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:40:27.577789 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.577754    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qw8m2"]
Sep 18 15:40:27.578050 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.578037    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerName="util"
Sep 18 15:40:27.578103 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.578051    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerName="util"
Sep 18 15:40:27.578103 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.578061    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerName="pull"
Sep 18 15:40:27.578103 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.578066    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerName="pull"
Sep 18 15:40:27.578103 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.578078    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerName="extract"
Sep 18 15:40:27.578103 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.578084    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerName="extract"
Sep 18 15:40:27.578268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.578127    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52" containerName="extract"
Sep 18 15:40:27.580804 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.580788    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:27.583439 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.583419    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 15:40:27.583661 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.583648    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-d5jqb\""
Sep 18 15:40:27.584563 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.584548    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 15:40:27.588722 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.588701    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qw8m2"]
Sep 18 15:40:27.671363 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.671328    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bxhj\" (UniqueName: \"kubernetes.io/projected/de77b589-5aef-43ec-94a9-6bd94bd27963-kube-api-access-6bxhj\") pod \"cert-manager-webhook-6486ffd997-qw8m2\" (UID: \"de77b589-5aef-43ec-94a9-6bd94bd27963\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:27.671500 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.671376    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de77b589-5aef-43ec-94a9-6bd94bd27963-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qw8m2\" (UID: \"de77b589-5aef-43ec-94a9-6bd94bd27963\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:27.771691 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.771659    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bxhj\" (UniqueName: \"kubernetes.io/projected/de77b589-5aef-43ec-94a9-6bd94bd27963-kube-api-access-6bxhj\") pod \"cert-manager-webhook-6486ffd997-qw8m2\" (UID: \"de77b589-5aef-43ec-94a9-6bd94bd27963\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:27.771872 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.771702    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de77b589-5aef-43ec-94a9-6bd94bd27963-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qw8m2\" (UID: \"de77b589-5aef-43ec-94a9-6bd94bd27963\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:27.781691 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.781642    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de77b589-5aef-43ec-94a9-6bd94bd27963-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qw8m2\" (UID: \"de77b589-5aef-43ec-94a9-6bd94bd27963\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:27.781884 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.781862    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bxhj\" (UniqueName: \"kubernetes.io/projected/de77b589-5aef-43ec-94a9-6bd94bd27963-kube-api-access-6bxhj\") pod \"cert-manager-webhook-6486ffd997-qw8m2\" (UID: \"de77b589-5aef-43ec-94a9-6bd94bd27963\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:27.889569 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.889529    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:27.901894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.901869    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj" event={"ID":"138a6cfd-4449-4ba2-8b09-c4ca1a9a6c52","Type":"ContainerDied","Data":"28c0b73ef6d431d8f2ee3777bcf44a1e53946b8f2c02970607af1a01b03b06fa"}
Sep 18 15:40:27.901894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.901884    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvvcvj"
Sep 18 15:40:27.902023 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:27.901913    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28c0b73ef6d431d8f2ee3777bcf44a1e53946b8f2c02970607af1a01b03b06fa"
Sep 18 15:40:28.161143 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:28.161121    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qw8m2"]
Sep 18 15:40:28.163428 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:40:28.163393    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde77b589_5aef_43ec_94a9_6bd94bd27963.slice/crio-dcb8d465dc597d8f8d99235c945a6ac7125464ae483b2b11d5607259cd38b392 WatchSource:0}: Error finding container dcb8d465dc597d8f8d99235c945a6ac7125464ae483b2b11d5607259cd38b392: Status 404 returned error can't find the container with id dcb8d465dc597d8f8d99235c945a6ac7125464ae483b2b11d5607259cd38b392
Sep 18 15:40:28.908759 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:28.908715    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2" event={"ID":"de77b589-5aef-43ec-94a9-6bd94bd27963","Type":"ContainerStarted","Data":"dcb8d465dc597d8f8d99235c945a6ac7125464ae483b2b11d5607259cd38b392"}
Sep 18 15:40:30.918721 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:30.918686    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2" event={"ID":"de77b589-5aef-43ec-94a9-6bd94bd27963","Type":"ContainerStarted","Data":"17cf31c3035ce53c919af973a2d2bdb3a6b393a788ee7eeae859861c87c307fb"}
Sep 18 15:40:30.919189 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:30.918732    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:30.939224 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:30.939171    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2" podStartSLOduration=1.3334468560000001 podStartE2EDuration="3.939157135s" totalImagesPullingTime="2.605710279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:40:30.936901064 +0000 UTC m=+355.595693726" watchObservedRunningTime="2026-09-18 15:40:30.939157135 +0000 UTC m=+355.597949794"
Sep 18 15:40:32.790367 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.790336    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fdr4d"]
Sep 18 15:40:32.793504 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.793487    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fdr4d"
Sep 18 15:40:32.796219 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.796198    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-n94cm\""
Sep 18 15:40:32.804902 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.804874    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6ggf\" (UniqueName: \"kubernetes.io/projected/9b733d65-5cca-4c41-924e-d2ecbede0ce0-kube-api-access-b6ggf\") pod \"cert-manager-55b4d954d5-fdr4d\" (UID: \"9b733d65-5cca-4c41-924e-d2ecbede0ce0\") " pod="cert-manager/cert-manager-55b4d954d5-fdr4d"
Sep 18 15:40:32.805016 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.804932    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b733d65-5cca-4c41-924e-d2ecbede0ce0-bound-sa-token\") pod \"cert-manager-55b4d954d5-fdr4d\" (UID: \"9b733d65-5cca-4c41-924e-d2ecbede0ce0\") " pod="cert-manager/cert-manager-55b4d954d5-fdr4d"
Sep 18 15:40:32.813503 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.813477    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fdr4d"]
Sep 18 15:40:32.905618 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.905590    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6ggf\" (UniqueName: \"kubernetes.io/projected/9b733d65-5cca-4c41-924e-d2ecbede0ce0-kube-api-access-b6ggf\") pod \"cert-manager-55b4d954d5-fdr4d\" (UID: \"9b733d65-5cca-4c41-924e-d2ecbede0ce0\") " pod="cert-manager/cert-manager-55b4d954d5-fdr4d"
Sep 18 15:40:32.905756 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.905640    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b733d65-5cca-4c41-924e-d2ecbede0ce0-bound-sa-token\") pod \"cert-manager-55b4d954d5-fdr4d\" (UID: \"9b733d65-5cca-4c41-924e-d2ecbede0ce0\") " pod="cert-manager/cert-manager-55b4d954d5-fdr4d"
Sep 18 15:40:32.920242 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.920216    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b733d65-5cca-4c41-924e-d2ecbede0ce0-bound-sa-token\") pod \"cert-manager-55b4d954d5-fdr4d\" (UID: \"9b733d65-5cca-4c41-924e-d2ecbede0ce0\") " pod="cert-manager/cert-manager-55b4d954d5-fdr4d"
Sep 18 15:40:32.920438 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:32.920418    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6ggf\" (UniqueName: \"kubernetes.io/projected/9b733d65-5cca-4c41-924e-d2ecbede0ce0-kube-api-access-b6ggf\") pod \"cert-manager-55b4d954d5-fdr4d\" (UID: \"9b733d65-5cca-4c41-924e-d2ecbede0ce0\") " pod="cert-manager/cert-manager-55b4d954d5-fdr4d"
Sep 18 15:40:33.102171 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:33.102090    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fdr4d"
Sep 18 15:40:33.376561 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:33.376489    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fdr4d"]
Sep 18 15:40:33.379598 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:40:33.379570    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b733d65_5cca_4c41_924e_d2ecbede0ce0.slice/crio-109222f7b430838108ca9f41514b8d44f5009169158e4b633057cee630471126 WatchSource:0}: Error finding container 109222f7b430838108ca9f41514b8d44f5009169158e4b633057cee630471126: Status 404 returned error can't find the container with id 109222f7b430838108ca9f41514b8d44f5009169158e4b633057cee630471126
Sep 18 15:40:33.932199 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:33.932167    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fdr4d" event={"ID":"9b733d65-5cca-4c41-924e-d2ecbede0ce0","Type":"ContainerStarted","Data":"e772df50871b160e6423f03827bff9e8d48bd24c4cdcfce5b8cd7817b42c34df"}
Sep 18 15:40:33.932553 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:33.932204    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fdr4d" event={"ID":"9b733d65-5cca-4c41-924e-d2ecbede0ce0","Type":"ContainerStarted","Data":"109222f7b430838108ca9f41514b8d44f5009169158e4b633057cee630471126"}
Sep 18 15:40:33.955365 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:33.955312    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-fdr4d" podStartSLOduration=1.955296344 podStartE2EDuration="1.955296344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:40:33.954312656 +0000 UTC m=+358.613105317" watchObservedRunningTime="2026-09-18 15:40:33.955296344 +0000 UTC m=+358.614089004"
Sep 18 15:40:34.224650 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.224577    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"]
Sep 18 15:40:34.227764 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.227748    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"
Sep 18 15:40:34.230382 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.230354    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-6vmcr\""
Sep 18 15:40:34.230382 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.230366    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 15:40:34.230531 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.230389    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 15:40:34.236440 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.236419    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"]
Sep 18 15:40:34.315149 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.315119    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvqfk\" (UniqueName: \"kubernetes.io/projected/352c8084-4ba7-4d0e-b17c-b419788ea8c6-kube-api-access-nvqfk\") pod \"openshift-lws-operator-fd8ccff4c-5qh2g\" (UID: \"352c8084-4ba7-4d0e-b17c-b419788ea8c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"
Sep 18 15:40:34.315303 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.315173    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/352c8084-4ba7-4d0e-b17c-b419788ea8c6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5qh2g\" (UID: \"352c8084-4ba7-4d0e-b17c-b419788ea8c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"
Sep 18 15:40:34.415655 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.415623    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/352c8084-4ba7-4d0e-b17c-b419788ea8c6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5qh2g\" (UID: \"352c8084-4ba7-4d0e-b17c-b419788ea8c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"
Sep 18 15:40:34.415831 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.415686    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvqfk\" (UniqueName: \"kubernetes.io/projected/352c8084-4ba7-4d0e-b17c-b419788ea8c6-kube-api-access-nvqfk\") pod \"openshift-lws-operator-fd8ccff4c-5qh2g\" (UID: \"352c8084-4ba7-4d0e-b17c-b419788ea8c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"
Sep 18 15:40:34.416054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.416035    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/352c8084-4ba7-4d0e-b17c-b419788ea8c6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5qh2g\" (UID: \"352c8084-4ba7-4d0e-b17c-b419788ea8c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"
Sep 18 15:40:34.424547 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.424526    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvqfk\" (UniqueName: \"kubernetes.io/projected/352c8084-4ba7-4d0e-b17c-b419788ea8c6-kube-api-access-nvqfk\") pod \"openshift-lws-operator-fd8ccff4c-5qh2g\" (UID: \"352c8084-4ba7-4d0e-b17c-b419788ea8c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"
Sep 18 15:40:34.537806 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.537779    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"
Sep 18 15:40:34.808493 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.808410    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g"]
Sep 18 15:40:34.812718 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:40:34.812682    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod352c8084_4ba7_4d0e_b17c_b419788ea8c6.slice/crio-630ae285e3384d55fd367baf4c6b49f656833e89546130a97b85c95c65e7953b WatchSource:0}: Error finding container 630ae285e3384d55fd367baf4c6b49f656833e89546130a97b85c95c65e7953b: Status 404 returned error can't find the container with id 630ae285e3384d55fd367baf4c6b49f656833e89546130a97b85c95c65e7953b
Sep 18 15:40:34.937825 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:34.937793    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g" event={"ID":"352c8084-4ba7-4d0e-b17c-b419788ea8c6","Type":"ContainerStarted","Data":"630ae285e3384d55fd367baf4c6b49f656833e89546130a97b85c95c65e7953b"}
Sep 18 15:40:36.925081 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:36.925054    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8m2"
Sep 18 15:40:36.951465 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:36.951430    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g" event={"ID":"352c8084-4ba7-4d0e-b17c-b419788ea8c6","Type":"ContainerStarted","Data":"ca365c9eaedb101cacb27fd583b1e2596651f987a1d79049d0a6c7b68b5e0131"}
Sep 18 15:40:36.968965 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:36.968923    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5qh2g" podStartSLOduration=0.904417438 podStartE2EDuration="2.968907249s" totalImagesPullingTime="2.064489811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:40:36.967723872 +0000 UTC m=+361.626516533" watchObservedRunningTime="2026-09-18 15:40:36.968907249 +0000 UTC m=+361.627699909"
Sep 18 15:40:39.607336 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.607304    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"]
Sep 18 15:40:39.610729 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.610712    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.613505 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.613482    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 15:40:39.614770 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.614752    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g427q\""
Sep 18 15:40:39.614770 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.614761    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 15:40:39.618930 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.618907    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"]
Sep 18 15:40:39.653302 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.653283    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.653400 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.653315    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6slp\" (UniqueName: \"kubernetes.io/projected/6efa1dd3-7dd5-42e3-bc90-30674f57638a-kube-api-access-k6slp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.653400 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.653388    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.753733 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.753702    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.753952 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.753738    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6slp\" (UniqueName: \"kubernetes.io/projected/6efa1dd3-7dd5-42e3-bc90-30674f57638a-kube-api-access-k6slp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.753952 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.753798    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.754181 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.754159    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.754243 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.754195    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.761645 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.761618    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6slp\" (UniqueName: \"kubernetes.io/projected/6efa1dd3-7dd5-42e3-bc90-30674f57638a-kube-api-access-k6slp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:39.919920 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:39.919826    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:40.282653 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:40.282617    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"]
Sep 18 15:40:40.285652 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:40:40.285624    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6efa1dd3_7dd5_42e3_bc90_30674f57638a.slice/crio-7c0f6a93a0bf83d6cf400314e4f90c4a060479f87d0b4447f480a47a03f562c2 WatchSource:0}: Error finding container 7c0f6a93a0bf83d6cf400314e4f90c4a060479f87d0b4447f480a47a03f562c2: Status 404 returned error can't find the container with id 7c0f6a93a0bf83d6cf400314e4f90c4a060479f87d0b4447f480a47a03f562c2
Sep 18 15:40:40.969786 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:40.969755    2585 generic.go:356] "Generic (PLEG): container finished" podID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerID="91dee4a507684109a632b36b3d99d75f99fbe250603885ad2beed5a7827c79cf" exitCode=0
Sep 18 15:40:40.970217 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:40.969805    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g" event={"ID":"6efa1dd3-7dd5-42e3-bc90-30674f57638a","Type":"ContainerDied","Data":"91dee4a507684109a632b36b3d99d75f99fbe250603885ad2beed5a7827c79cf"}
Sep 18 15:40:40.970217 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:40.969895    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g" event={"ID":"6efa1dd3-7dd5-42e3-bc90-30674f57638a","Type":"ContainerStarted","Data":"7c0f6a93a0bf83d6cf400314e4f90c4a060479f87d0b4447f480a47a03f562c2"}
Sep 18 15:40:41.975441 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:41.975357    2585 generic.go:356] "Generic (PLEG): container finished" podID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerID="5ca1bb0b5e167b8e27e90e7b04dcfceb30e404ab576a75d91dce610e6213c0ad" exitCode=0
Sep 18 15:40:41.975441 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:41.975403    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g" event={"ID":"6efa1dd3-7dd5-42e3-bc90-30674f57638a","Type":"ContainerDied","Data":"5ca1bb0b5e167b8e27e90e7b04dcfceb30e404ab576a75d91dce610e6213c0ad"}
Sep 18 15:40:42.981658 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:42.981619    2585 generic.go:356] "Generic (PLEG): container finished" podID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerID="ba1f109137bcc35cbcb6ea02c4484253f46d0b04432bab6c3d76e8774b58fb3c" exitCode=0
Sep 18 15:40:42.982114 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:42.981707    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g" event={"ID":"6efa1dd3-7dd5-42e3-bc90-30674f57638a","Type":"ContainerDied","Data":"ba1f109137bcc35cbcb6ea02c4484253f46d0b04432bab6c3d76e8774b58fb3c"}
Sep 18 15:40:44.100905 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.100884    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:44.190326 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.190298    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-util\") pod \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") "
Sep 18 15:40:44.190490 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.190339    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-bundle\") pod \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") "
Sep 18 15:40:44.190490 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.190382    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6slp\" (UniqueName: \"kubernetes.io/projected/6efa1dd3-7dd5-42e3-bc90-30674f57638a-kube-api-access-k6slp\") pod \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\" (UID: \"6efa1dd3-7dd5-42e3-bc90-30674f57638a\") "
Sep 18 15:40:44.191052 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.191026    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-bundle" (OuterVolumeSpecName: "bundle") pod "6efa1dd3-7dd5-42e3-bc90-30674f57638a" (UID: "6efa1dd3-7dd5-42e3-bc90-30674f57638a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:40:44.192350 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.192330    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6efa1dd3-7dd5-42e3-bc90-30674f57638a-kube-api-access-k6slp" (OuterVolumeSpecName: "kube-api-access-k6slp") pod "6efa1dd3-7dd5-42e3-bc90-30674f57638a" (UID: "6efa1dd3-7dd5-42e3-bc90-30674f57638a"). InnerVolumeSpecName "kube-api-access-k6slp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:40:44.196053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.196016    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-util" (OuterVolumeSpecName: "util") pod "6efa1dd3-7dd5-42e3-bc90-30674f57638a" (UID: "6efa1dd3-7dd5-42e3-bc90-30674f57638a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:40:44.291263 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.291189    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6slp\" (UniqueName: \"kubernetes.io/projected/6efa1dd3-7dd5-42e3-bc90-30674f57638a-kube-api-access-k6slp\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:40:44.291263 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.291212    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-util\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:40:44.291263 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.291221    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa1dd3-7dd5-42e3-bc90-30674f57638a-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:40:44.997057 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.997028    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g"
Sep 18 15:40:44.997057 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.997033    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff8p5g" event={"ID":"6efa1dd3-7dd5-42e3-bc90-30674f57638a","Type":"ContainerDied","Data":"7c0f6a93a0bf83d6cf400314e4f90c4a060479f87d0b4447f480a47a03f562c2"}
Sep 18 15:40:44.997278 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:44.997107    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c0f6a93a0bf83d6cf400314e4f90c4a060479f87d0b4447f480a47a03f562c2"
Sep 18 15:40:56.260058 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.260019    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"]
Sep 18 15:40:56.261152 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.261127    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerName="pull"
Sep 18 15:40:56.261317 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.261306    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerName="pull"
Sep 18 15:40:56.261419 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.261410    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerName="extract"
Sep 18 15:40:56.261501 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.261491    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerName="extract"
Sep 18 15:40:56.261721 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.261702    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerName="util"
Sep 18 15:40:56.261721 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.261716    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerName="util"
Sep 18 15:40:56.261931 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.261919    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6efa1dd3-7dd5-42e3-bc90-30674f57638a" containerName="extract"
Sep 18 15:40:56.268411 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.268393    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.271383 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.271357    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"]
Sep 18 15:40:56.276958 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.276939    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 15:40:56.277045 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.277026    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 15:40:56.278032 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.278014    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g427q\""
Sep 18 15:40:56.373135 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.373102    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.373291 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.373151    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86kdl\" (UniqueName: \"kubernetes.io/projected/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-kube-api-access-86kdl\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.373291 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.373182    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.474200 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.474167    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.474388 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.474212    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86kdl\" (UniqueName: \"kubernetes.io/projected/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-kube-api-access-86kdl\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.474388 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.474252    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.474536 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.474517    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.474599 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.474576    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.490308 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.490274    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86kdl\" (UniqueName: \"kubernetes.io/projected/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-kube-api-access-86kdl\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.578657 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.578588    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:40:56.862892 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:56.862806    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"]
Sep 18 15:40:56.865891 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:40:56.865860    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538ac3a9_4a5f_4454_bb6e_1a19110ed5ed.slice/crio-057b3fbc7bcdfd4ab3c2b0a3f8388b1d26c062c0a7ab13674472ec16ed50647f WatchSource:0}: Error finding container 057b3fbc7bcdfd4ab3c2b0a3f8388b1d26c062c0a7ab13674472ec16ed50647f: Status 404 returned error can't find the container with id 057b3fbc7bcdfd4ab3c2b0a3f8388b1d26c062c0a7ab13674472ec16ed50647f
Sep 18 15:40:57.049576 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.049539    2585 generic.go:356] "Generic (PLEG): container finished" podID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerID="b59f57e0ff26e90cedb1bb05415e9e22e27e74083d8af692539338139204dacd" exitCode=0
Sep 18 15:40:57.049735 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.049630    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8" event={"ID":"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed","Type":"ContainerDied","Data":"b59f57e0ff26e90cedb1bb05415e9e22e27e74083d8af692539338139204dacd"}
Sep 18 15:40:57.049735 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.049671    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8" event={"ID":"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed","Type":"ContainerStarted","Data":"057b3fbc7bcdfd4ab3c2b0a3f8388b1d26c062c0a7ab13674472ec16ed50647f"}
Sep 18 15:40:57.443694 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.443656    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"]
Sep 18 15:40:57.447069 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.447048    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.451532 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.451513    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 15:40:57.451998 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.451975    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cqmrf\""
Sep 18 15:40:57.451998 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.451991    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 15:40:57.452170 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.452003    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 15:40:57.452170 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.452127    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 15:40:57.482015 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.481994    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r886h\" (UniqueName: \"kubernetes.io/projected/73c5053f-5d25-4dc2-83b8-62feafa2ead4-kube-api-access-r886h\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.482125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.482048    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/73c5053f-5d25-4dc2-83b8-62feafa2ead4-manifests\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.482125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.482098    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73c5053f-5d25-4dc2-83b8-62feafa2ead4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.482225 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.482136    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73c5053f-5d25-4dc2-83b8-62feafa2ead4-webhook-cert\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.482225 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.482168    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73c5053f-5d25-4dc2-83b8-62feafa2ead4-tmp\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.484992 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.484955    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"]
Sep 18 15:40:57.583017 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.582986    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73c5053f-5d25-4dc2-83b8-62feafa2ead4-tmp\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.583163 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.583032    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r886h\" (UniqueName: \"kubernetes.io/projected/73c5053f-5d25-4dc2-83b8-62feafa2ead4-kube-api-access-r886h\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.583163 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.583075    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/73c5053f-5d25-4dc2-83b8-62feafa2ead4-manifests\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.583163 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.583089    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73c5053f-5d25-4dc2-83b8-62feafa2ead4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.583163 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.583112    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73c5053f-5d25-4dc2-83b8-62feafa2ead4-webhook-cert\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.583488 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.583465    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/73c5053f-5d25-4dc2-83b8-62feafa2ead4-manifests\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.585217 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.585185    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73c5053f-5d25-4dc2-83b8-62feafa2ead4-tmp\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.585422 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.585402    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73c5053f-5d25-4dc2-83b8-62feafa2ead4-webhook-cert\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.585554 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.585537    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73c5053f-5d25-4dc2-83b8-62feafa2ead4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.592572 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.592553    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r886h\" (UniqueName: \"kubernetes.io/projected/73c5053f-5d25-4dc2-83b8-62feafa2ead4-kube-api-access-r886h\") pod \"opendatahub-operator-controller-manager-54b8c8d665-68hsx\" (UID: \"73c5053f-5d25-4dc2-83b8-62feafa2ead4\") " pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:57.757611 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:57.757590    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:40:58.034356 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:58.034272    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"]
Sep 18 15:40:58.036791 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:40:58.036763    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c5053f_5d25_4dc2_83b8_62feafa2ead4.slice/crio-d1777ddb7d0e92f7469a1f7c6b85dd2ffbecce12f95c81a013eeb5e53353fa5f WatchSource:0}: Error finding container d1777ddb7d0e92f7469a1f7c6b85dd2ffbecce12f95c81a013eeb5e53353fa5f: Status 404 returned error can't find the container with id d1777ddb7d0e92f7469a1f7c6b85dd2ffbecce12f95c81a013eeb5e53353fa5f
Sep 18 15:40:58.055435 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:58.055408    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx" event={"ID":"73c5053f-5d25-4dc2-83b8-62feafa2ead4","Type":"ContainerStarted","Data":"d1777ddb7d0e92f7469a1f7c6b85dd2ffbecce12f95c81a013eeb5e53353fa5f"}
Sep 18 15:40:58.057164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:58.057143    2585 generic.go:356] "Generic (PLEG): container finished" podID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerID="854ad35ffa90209026fc47ea242a2fcdf98ec19f15bd897ed2345465888aa888" exitCode=0
Sep 18 15:40:58.057266 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:58.057226    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8" event={"ID":"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed","Type":"ContainerDied","Data":"854ad35ffa90209026fc47ea242a2fcdf98ec19f15bd897ed2345465888aa888"}
Sep 18 15:40:59.066073 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:59.066037    2585 generic.go:356] "Generic (PLEG): container finished" podID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerID="d1107e54b87d9b0d3c740ccc7bae3e12499ba8393895f2895b1253dfbce46a9e" exitCode=0
Sep 18 15:40:59.066690 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:40:59.066114    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8" event={"ID":"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed","Type":"ContainerDied","Data":"d1107e54b87d9b0d3c740ccc7bae3e12499ba8393895f2895b1253dfbce46a9e"}
Sep 18 15:41:00.959502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:00.959475    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:41:01.011063 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.011041    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86kdl\" (UniqueName: \"kubernetes.io/projected/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-kube-api-access-86kdl\") pod \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") "
Sep 18 15:41:01.011144 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.011103    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-bundle\") pod \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") "
Sep 18 15:41:01.011189 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.011161    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-util\") pod \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\" (UID: \"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed\") "
Sep 18 15:41:01.012004 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.011975    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-bundle" (OuterVolumeSpecName: "bundle") pod "538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" (UID: "538ac3a9-4a5f-4454-bb6e-1a19110ed5ed"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:41:01.013184 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.013160    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-kube-api-access-86kdl" (OuterVolumeSpecName: "kube-api-access-86kdl") pod "538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" (UID: "538ac3a9-4a5f-4454-bb6e-1a19110ed5ed"). InnerVolumeSpecName "kube-api-access-86kdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:41:01.016426 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.016393    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-util" (OuterVolumeSpecName: "util") pod "538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" (UID: "538ac3a9-4a5f-4454-bb6e-1a19110ed5ed"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:41:01.077941 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.077910    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8"
Sep 18 15:41:01.078096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.077930    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b089ms8" event={"ID":"538ac3a9-4a5f-4454-bb6e-1a19110ed5ed","Type":"ContainerDied","Data":"057b3fbc7bcdfd4ab3c2b0a3f8388b1d26c062c0a7ab13674472ec16ed50647f"}
Sep 18 15:41:01.078096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.078016    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="057b3fbc7bcdfd4ab3c2b0a3f8388b1d26c062c0a7ab13674472ec16ed50647f"
Sep 18 15:41:01.079572 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.079543    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx" event={"ID":"73c5053f-5d25-4dc2-83b8-62feafa2ead4","Type":"ContainerStarted","Data":"e08a12231ac71a25ed242a12e390b9aadbed97bcdf50a319287259cc8bcc9649"}
Sep 18 15:41:01.112035 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.112006    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-util\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:41:01.112035 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.112034    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86kdl\" (UniqueName: \"kubernetes.io/projected/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-kube-api-access-86kdl\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:41:01.112200 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:01.112049    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/538ac3a9-4a5f-4454-bb6e-1a19110ed5ed-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:41:03.090557 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:03.090526    2585 generic.go:356] "Generic (PLEG): container finished" podID="73c5053f-5d25-4dc2-83b8-62feafa2ead4" containerID="e08a12231ac71a25ed242a12e390b9aadbed97bcdf50a319287259cc8bcc9649" exitCode=0
Sep 18 15:41:03.090928 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:03.090597    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx" event={"ID":"73c5053f-5d25-4dc2-83b8-62feafa2ead4","Type":"ContainerDied","Data":"e08a12231ac71a25ed242a12e390b9aadbed97bcdf50a319287259cc8bcc9649"}
Sep 18 15:41:04.097321 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:04.097286    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx" event={"ID":"73c5053f-5d25-4dc2-83b8-62feafa2ead4","Type":"ContainerStarted","Data":"e7861fd14e510c6ed561ca2d278afd5dd0725642edcd1d23f5136e2c7133077b"}
Sep 18 15:41:04.097820 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:04.097503    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:41:04.124079 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:04.124029    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx" podStartSLOduration=2.768431908 podStartE2EDuration="7.124014727s" totalImagesPullingTime="3.29998128s" totalInitContainerRuntime="1.055601539s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:40:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:41:04.121922724 +0000 UTC m=+388.780715386" watchObservedRunningTime="2026-09-18 15:41:04.124014727 +0000 UTC m=+388.782807387"
Sep 18 15:41:15.103817 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:15.103780    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54b8c8d665-68hsx"
Sep 18 15:41:17.719795 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.719764    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"]
Sep 18 15:41:17.720201 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.720052    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerName="pull"
Sep 18 15:41:17.720201 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.720063    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerName="pull"
Sep 18 15:41:17.720201 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.720079    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerName="util"
Sep 18 15:41:17.720201 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.720084    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerName="util"
Sep 18 15:41:17.720201 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.720093    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerName="extract"
Sep 18 15:41:17.720201 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.720099    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerName="extract"
Sep 18 15:41:17.720201 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.720151    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="538ac3a9-4a5f-4454-bb6e-1a19110ed5ed" containerName="extract"
Sep 18 15:41:17.728176 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.728151    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:17.731714 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.731684    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 15:41:17.731867 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.731799    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gw7mr\""
Sep 18 15:41:17.732732 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.732713    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 15:41:17.732801 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.732716    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 15:41:17.732851 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.732716    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 15:41:17.743544 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.743521    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"]
Sep 18 15:41:17.842121 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.842081    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a82478e-701b-45ba-9d23-4eb836cc9f52-tls-certs\") pod \"kube-auth-proxy-7fbb59494c-8wkbn\" (UID: \"7a82478e-701b-45ba-9d23-4eb836cc9f52\") " pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:17.842312 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.842126    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a82478e-701b-45ba-9d23-4eb836cc9f52-tmp\") pod \"kube-auth-proxy-7fbb59494c-8wkbn\" (UID: \"7a82478e-701b-45ba-9d23-4eb836cc9f52\") " pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:17.842312 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.842176    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh8dt\" (UniqueName: \"kubernetes.io/projected/7a82478e-701b-45ba-9d23-4eb836cc9f52-kube-api-access-kh8dt\") pod \"kube-auth-proxy-7fbb59494c-8wkbn\" (UID: \"7a82478e-701b-45ba-9d23-4eb836cc9f52\") " pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:17.943160 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.943126    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a82478e-701b-45ba-9d23-4eb836cc9f52-tls-certs\") pod \"kube-auth-proxy-7fbb59494c-8wkbn\" (UID: \"7a82478e-701b-45ba-9d23-4eb836cc9f52\") " pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:17.943160 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.943162    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a82478e-701b-45ba-9d23-4eb836cc9f52-tmp\") pod \"kube-auth-proxy-7fbb59494c-8wkbn\" (UID: \"7a82478e-701b-45ba-9d23-4eb836cc9f52\") " pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:17.943429 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.943186    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh8dt\" (UniqueName: \"kubernetes.io/projected/7a82478e-701b-45ba-9d23-4eb836cc9f52-kube-api-access-kh8dt\") pod \"kube-auth-proxy-7fbb59494c-8wkbn\" (UID: \"7a82478e-701b-45ba-9d23-4eb836cc9f52\") " pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:17.945576 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.945550    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a82478e-701b-45ba-9d23-4eb836cc9f52-tmp\") pod \"kube-auth-proxy-7fbb59494c-8wkbn\" (UID: \"7a82478e-701b-45ba-9d23-4eb836cc9f52\") " pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:17.945715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.945700    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a82478e-701b-45ba-9d23-4eb836cc9f52-tls-certs\") pod \"kube-auth-proxy-7fbb59494c-8wkbn\" (UID: \"7a82478e-701b-45ba-9d23-4eb836cc9f52\") " pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:17.954166 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:17.954142    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh8dt\" (UniqueName: \"kubernetes.io/projected/7a82478e-701b-45ba-9d23-4eb836cc9f52-kube-api-access-kh8dt\") pod \"kube-auth-proxy-7fbb59494c-8wkbn\" (UID: \"7a82478e-701b-45ba-9d23-4eb836cc9f52\") " pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:18.037219 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:18.037184    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"
Sep 18 15:41:18.328904 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:18.328878    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn"]
Sep 18 15:41:18.330582 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:41:18.330553    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a82478e_701b_45ba_9d23_4eb836cc9f52.slice/crio-6fb758a92b926b8746042cf1419b0caedf2976d0f68e576550db5d26c29bde54 WatchSource:0}: Error finding container 6fb758a92b926b8746042cf1419b0caedf2976d0f68e576550db5d26c29bde54: Status 404 returned error can't find the container with id 6fb758a92b926b8746042cf1419b0caedf2976d0f68e576550db5d26c29bde54
Sep 18 15:41:19.164725 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:19.164692    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn" event={"ID":"7a82478e-701b-45ba-9d23-4eb836cc9f52","Type":"ContainerStarted","Data":"6fb758a92b926b8746042cf1419b0caedf2976d0f68e576550db5d26c29bde54"}
Sep 18 15:41:22.180989 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:22.180947    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn" event={"ID":"7a82478e-701b-45ba-9d23-4eb836cc9f52","Type":"ContainerStarted","Data":"949e707e73b9a6a7e2fb2b3c60097104a63a27d10fe998941c0571339da2ed2d"}
Sep 18 15:41:22.204780 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:22.204734    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7fbb59494c-8wkbn" podStartSLOduration=2.038922441 podStartE2EDuration="5.204718303s" totalImagesPullingTime="3.165795862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:41:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:41:22.203196216 +0000 UTC m=+406.861988878" watchObservedRunningTime="2026-09-18 15:41:22.204718303 +0000 UTC m=+406.863510962"
Sep 18 15:41:25.088879 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.088817    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"]
Sep 18 15:41:25.092137 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.092119    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.095537 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.095509    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 15:41:25.096937 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.096893    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7lxc5\""
Sep 18 15:41:25.105488 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.105466    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"]
Sep 18 15:41:25.198613 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.198582    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.198613 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.198619    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.198829 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.198637    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ecb7141-4a46-45f5-ad03-18f347393040-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.198829 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.198731    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.198829 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.198759    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ecb7141-4a46-45f5-ad03-18f347393040-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.198829 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.198780    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ecb7141-4a46-45f5-ad03-18f347393040-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.198829 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.198806    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb56t\" (UniqueName: \"kubernetes.io/projected/3ecb7141-4a46-45f5-ad03-18f347393040-kube-api-access-rb56t\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.199014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.198868    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.199014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.198888    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.299455 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.299424    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.299722 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.299701    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.299892 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.299877    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.300033 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.300019    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.300146 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.300133    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ecb7141-4a46-45f5-ad03-18f347393040-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.300327 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.300312    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.300439 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.300425    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ecb7141-4a46-45f5-ad03-18f347393040-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.300552 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.300540    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ecb7141-4a46-45f5-ad03-18f347393040-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.300664 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.300648    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb56t\" (UniqueName: \"kubernetes.io/projected/3ecb7141-4a46-45f5-ad03-18f347393040-kube-api-access-rb56t\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.301603 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.301582    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.302061 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.302045    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.302430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.302414    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.303109 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.303090    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.303825 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.303805    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ecb7141-4a46-45f5-ad03-18f347393040-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.307575 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.307544    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ecb7141-4a46-45f5-ad03-18f347393040-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.312107 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.312063    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb56t\" (UniqueName: \"kubernetes.io/projected/3ecb7141-4a46-45f5-ad03-18f347393040-kube-api-access-rb56t\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.312190 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.312121    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ecb7141-4a46-45f5-ad03-18f347393040-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.318086 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.318066    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ecb7141-4a46-45f5-ad03-18f347393040-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48\" (UID: \"3ecb7141-4a46-45f5-ad03-18f347393040\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.401823 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.401732    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:25.677653 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:25.677630    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"]
Sep 18 15:41:25.679440 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:41:25.679417    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ecb7141_4a46_45f5_ad03_18f347393040.slice/crio-701216902dc3fe53b051944fe4fb7fda159b2dfbea2367dd47b9144ea64b774e WatchSource:0}: Error finding container 701216902dc3fe53b051944fe4fb7fda159b2dfbea2367dd47b9144ea64b774e: Status 404 returned error can't find the container with id 701216902dc3fe53b051944fe4fb7fda159b2dfbea2367dd47b9144ea64b774e
Sep 18 15:41:26.199290 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:26.199253    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48" event={"ID":"3ecb7141-4a46-45f5-ad03-18f347393040","Type":"ContainerStarted","Data":"701216902dc3fe53b051944fe4fb7fda159b2dfbea2367dd47b9144ea64b774e"}
Sep 18 15:41:28.141253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.141203    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:41:28.141554 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.141291    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:41:28.141554 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.141317    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:41:28.158616 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.158591    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lsph5"]
Sep 18 15:41:28.163068 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.163047    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:28.165609 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.165589    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 18 15:41:28.165705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.165686    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tdqjr\""
Sep 18 15:41:28.172258 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.172233    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lsph5"]
Sep 18 15:41:28.210384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.210353    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48" event={"ID":"3ecb7141-4a46-45f5-ad03-18f347393040","Type":"ContainerStarted","Data":"4810228c9aab5dcddaf95ecbeb50e5962e1511f21ac29fb10b223356169a136d"}
Sep 18 15:41:28.229268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.229222    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10a9b66-543d-42eb-a912-7d1efeb41da1-cert\") pod \"odh-model-controller-7998c67dc4-lsph5\" (UID: \"e10a9b66-543d-42eb-a912-7d1efeb41da1\") " pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:28.229452 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.229286    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87p7v\" (UniqueName: \"kubernetes.io/projected/e10a9b66-543d-42eb-a912-7d1efeb41da1-kube-api-access-87p7v\") pod \"odh-model-controller-7998c67dc4-lsph5\" (UID: \"e10a9b66-543d-42eb-a912-7d1efeb41da1\") " pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:28.238287 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.238226    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48" podStartSLOduration=0.778739147 podStartE2EDuration="3.238206056s" totalImagesPullingTime="2.459466909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:41:28.23602919 +0000 UTC m=+412.894821849" watchObservedRunningTime="2026-09-18 15:41:28.238206056 +0000 UTC m=+412.896998770"
Sep 18 15:41:28.330751 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.330710    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10a9b66-543d-42eb-a912-7d1efeb41da1-cert\") pod \"odh-model-controller-7998c67dc4-lsph5\" (UID: \"e10a9b66-543d-42eb-a912-7d1efeb41da1\") " pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:28.330929 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.330770    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87p7v\" (UniqueName: \"kubernetes.io/projected/e10a9b66-543d-42eb-a912-7d1efeb41da1-kube-api-access-87p7v\") pod \"odh-model-controller-7998c67dc4-lsph5\" (UID: \"e10a9b66-543d-42eb-a912-7d1efeb41da1\") " pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:28.331110 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:41:28.331088    2585 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 18 15:41:28.331186 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:41:28.331161    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10a9b66-543d-42eb-a912-7d1efeb41da1-cert podName:e10a9b66-543d-42eb-a912-7d1efeb41da1 nodeName:}" failed. No retries permitted until 2026-09-18 15:41:28.831142579 +0000 UTC m=+413.489935218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e10a9b66-543d-42eb-a912-7d1efeb41da1-cert") pod "odh-model-controller-7998c67dc4-lsph5" (UID: "e10a9b66-543d-42eb-a912-7d1efeb41da1") : secret "odh-model-controller-webhook-cert" not found
Sep 18 15:41:28.342427 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.342397    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87p7v\" (UniqueName: \"kubernetes.io/projected/e10a9b66-543d-42eb-a912-7d1efeb41da1-kube-api-access-87p7v\") pod \"odh-model-controller-7998c67dc4-lsph5\" (UID: \"e10a9b66-543d-42eb-a912-7d1efeb41da1\") " pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:28.402243 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.402159    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:28.834423 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.834385    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10a9b66-543d-42eb-a912-7d1efeb41da1-cert\") pod \"odh-model-controller-7998c67dc4-lsph5\" (UID: \"e10a9b66-543d-42eb-a912-7d1efeb41da1\") " pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:28.836707 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.836684    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10a9b66-543d-42eb-a912-7d1efeb41da1-cert\") pod \"odh-model-controller-7998c67dc4-lsph5\" (UID: \"e10a9b66-543d-42eb-a912-7d1efeb41da1\") " pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:28.850281 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.850250    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wxcsp"]
Sep 18 15:41:28.853520 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.853505    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:41:28.855982 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.855964    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qdfz9\""
Sep 18 15:41:28.856092 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.855999    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 18 15:41:28.864698 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.864674    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wxcsp"]
Sep 18 15:41:28.935986 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.935939    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ca7bdbe-7481-4158-ad65-cdbdd278a354-tls-certs\") pod \"model-serving-api-54d89bd5fc-wxcsp\" (UID: \"7ca7bdbe-7481-4158-ad65-cdbdd278a354\") " pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:41:28.936175 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:28.936067    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtdnm\" (UniqueName: \"kubernetes.io/projected/7ca7bdbe-7481-4158-ad65-cdbdd278a354-kube-api-access-rtdnm\") pod \"model-serving-api-54d89bd5fc-wxcsp\" (UID: \"7ca7bdbe-7481-4158-ad65-cdbdd278a354\") " pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:41:29.036438 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:29.036404    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ca7bdbe-7481-4158-ad65-cdbdd278a354-tls-certs\") pod \"model-serving-api-54d89bd5fc-wxcsp\" (UID: \"7ca7bdbe-7481-4158-ad65-cdbdd278a354\") " pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:41:29.036595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:29.036466    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtdnm\" (UniqueName: \"kubernetes.io/projected/7ca7bdbe-7481-4158-ad65-cdbdd278a354-kube-api-access-rtdnm\") pod \"model-serving-api-54d89bd5fc-wxcsp\" (UID: \"7ca7bdbe-7481-4158-ad65-cdbdd278a354\") " pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:41:29.038726 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:29.038697    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ca7bdbe-7481-4158-ad65-cdbdd278a354-tls-certs\") pod \"model-serving-api-54d89bd5fc-wxcsp\" (UID: \"7ca7bdbe-7481-4158-ad65-cdbdd278a354\") " pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:41:29.045421 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:29.045397    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtdnm\" (UniqueName: \"kubernetes.io/projected/7ca7bdbe-7481-4158-ad65-cdbdd278a354-kube-api-access-rtdnm\") pod \"model-serving-api-54d89bd5fc-wxcsp\" (UID: \"7ca7bdbe-7481-4158-ad65-cdbdd278a354\") " pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:41:29.088441 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:29.088367    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:29.164342 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:29.164311    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:41:29.355291 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:29.355266    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lsph5"]
Sep 18 15:41:29.357338 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:41:29.357307    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10a9b66_543d_42eb_a912_7d1efeb41da1.slice/crio-c17bb13a25b5ae86781ca6b331929a5b4f20f73331ac24ec2f42f46f3e942003 WatchSource:0}: Error finding container c17bb13a25b5ae86781ca6b331929a5b4f20f73331ac24ec2f42f46f3e942003: Status 404 returned error can't find the container with id c17bb13a25b5ae86781ca6b331929a5b4f20f73331ac24ec2f42f46f3e942003
Sep 18 15:41:29.407090 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:29.407055    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:29.434412 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:29.434381    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wxcsp"]
Sep 18 15:41:29.438148 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:41:29.438122    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca7bdbe_7481_4158_ad65_cdbdd278a354.slice/crio-261869744c80c06cd5c5513ce141d3270a6fc63a8e53af1aebd35533e3083144 WatchSource:0}: Error finding container 261869744c80c06cd5c5513ce141d3270a6fc63a8e53af1aebd35533e3083144: Status 404 returned error can't find the container with id 261869744c80c06cd5c5513ce141d3270a6fc63a8e53af1aebd35533e3083144
Sep 18 15:41:30.225521 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:30.225478    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp" event={"ID":"7ca7bdbe-7481-4158-ad65-cdbdd278a354","Type":"ContainerStarted","Data":"261869744c80c06cd5c5513ce141d3270a6fc63a8e53af1aebd35533e3083144"}
Sep 18 15:41:30.227078 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:30.227051    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lsph5" event={"ID":"e10a9b66-543d-42eb-a912-7d1efeb41da1","Type":"ContainerStarted","Data":"c17bb13a25b5ae86781ca6b331929a5b4f20f73331ac24ec2f42f46f3e942003"}
Sep 18 15:41:30.227722 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:30.227405    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:30.228405 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:30.228385    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48"
Sep 18 15:41:33.247340 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:33.247307    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp" event={"ID":"7ca7bdbe-7481-4158-ad65-cdbdd278a354","Type":"ContainerStarted","Data":"6c1f8ab66a215df211d6ea82f309ec0c0972674e7a7543f61426184cb1e96ccd"}
Sep 18 15:41:33.247752 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:33.247429    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:41:33.249096 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:33.249070    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lsph5" event={"ID":"e10a9b66-543d-42eb-a912-7d1efeb41da1","Type":"ContainerStarted","Data":"ccee9f2c424b7ee3bacf8e4fab72870fe6a250ff133cfc42beb66930bf59368f"}
Sep 18 15:41:33.249212 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:33.249119    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:33.271457 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:33.271409    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp" podStartSLOduration=2.226235556 podStartE2EDuration="5.271391101s" totalImagesPullingTime="3.045155545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:41:33.268925317 +0000 UTC m=+417.927717977" watchObservedRunningTime="2026-09-18 15:41:33.271391101 +0000 UTC m=+417.930183760"
Sep 18 15:41:33.293994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:33.293933    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-lsph5" podStartSLOduration=1.9936708109999999 podStartE2EDuration="5.293918059s" totalImagesPullingTime="3.300247248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:41:33.292316167 +0000 UTC m=+417.951108827" watchObservedRunningTime="2026-09-18 15:41:33.293918059 +0000 UTC m=+417.952710753"
Sep 18 15:41:35.229241 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.229204    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"]
Sep 18 15:41:35.231595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.231573    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.236784 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.236765    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 15:41:35.237127 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.237111    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8s7zz\""
Sep 18 15:41:35.237222 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.237166    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 15:41:35.237704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.237682    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 15:41:35.260428 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.260408    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"]
Sep 18 15:41:35.393652 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.393621    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d22ec279-6ea8-49f0-9524-548a030ef448-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.393867 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.393658    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d22ec279-6ea8-49f0-9524-548a030ef448-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.393867 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.393698    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d22ec279-6ea8-49f0-9524-548a030ef448-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.393867 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.393752    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d22ec279-6ea8-49f0-9524-548a030ef448-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.393867 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.393795    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d22ec279-6ea8-49f0-9524-548a030ef448-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.393867 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.393819    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhjmn\" (UniqueName: \"kubernetes.io/projected/d22ec279-6ea8-49f0-9524-548a030ef448-kube-api-access-qhjmn\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.394087 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.393877    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d22ec279-6ea8-49f0-9524-548a030ef448-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.494803 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.494763    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d22ec279-6ea8-49f0-9524-548a030ef448-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.494803 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.494806    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d22ec279-6ea8-49f0-9524-548a030ef448-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.495032 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.494830    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d22ec279-6ea8-49f0-9524-548a030ef448-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.495032 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.494880    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhjmn\" (UniqueName: \"kubernetes.io/projected/d22ec279-6ea8-49f0-9524-548a030ef448-kube-api-access-qhjmn\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.495032 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.494910    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d22ec279-6ea8-49f0-9524-548a030ef448-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.495032 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.494977    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d22ec279-6ea8-49f0-9524-548a030ef448-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.495032 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.495016    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d22ec279-6ea8-49f0-9524-548a030ef448-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.495668 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.495641    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d22ec279-6ea8-49f0-9524-548a030ef448-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.497298 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.497271    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d22ec279-6ea8-49f0-9524-548a030ef448-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.497399 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.497328    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d22ec279-6ea8-49f0-9524-548a030ef448-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.497570 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.497553    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d22ec279-6ea8-49f0-9524-548a030ef448-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.497630 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.497606    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d22ec279-6ea8-49f0-9524-548a030ef448-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.503366 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.503340    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d22ec279-6ea8-49f0-9524-548a030ef448-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.504339 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.504316    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhjmn\" (UniqueName: \"kubernetes.io/projected/d22ec279-6ea8-49f0-9524-548a030ef448-kube-api-access-qhjmn\") pod \"istiod-openshift-gateway-5677ccb49b-k589d\" (UID: \"d22ec279-6ea8-49f0-9524-548a030ef448\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.540799 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.540777    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:35.829540 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:35.829471    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"]
Sep 18 15:41:35.831932 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:41:35.831903    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22ec279_6ea8_49f0_9524_548a030ef448.slice/crio-9023cec3e560cb0a978a40c7b461339a5dcd326b377de02aba0143736de44f61 WatchSource:0}: Error finding container 9023cec3e560cb0a978a40c7b461339a5dcd326b377de02aba0143736de44f61: Status 404 returned error can't find the container with id 9023cec3e560cb0a978a40c7b461339a5dcd326b377de02aba0143736de44f61
Sep 18 15:41:36.264281 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:36.264245    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d" event={"ID":"d22ec279-6ea8-49f0-9524-548a030ef448","Type":"ContainerStarted","Data":"9023cec3e560cb0a978a40c7b461339a5dcd326b377de02aba0143736de44f61"}
Sep 18 15:41:38.052474 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:38.052440    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:41:38.052756 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:38.052508    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:41:38.277516 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:38.277475    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d" event={"ID":"d22ec279-6ea8-49f0-9524-548a030ef448","Type":"ContainerStarted","Data":"f903377a4e8a202a0e5a7c5e8355395357703767f45352e0ba6fc0b6d9a24aa1"}
Sep 18 15:41:38.277718 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:38.277556    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:38.316019 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:38.315918    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d" podStartSLOduration=1.098023498 podStartE2EDuration="3.315900546s" totalImagesPullingTime="2.217877048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:41:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:41:38.314982672 +0000 UTC m=+422.973775335" watchObservedRunningTime="2026-09-18 15:41:38.315900546 +0000 UTC m=+422.974693206"
Sep 18 15:41:39.283829 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:39.283802    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k589d"
Sep 18 15:41:44.256726 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:44.256696    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-lsph5"
Sep 18 15:41:44.258676 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:41:44.258658    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-wxcsp"
Sep 18 15:42:33.612406 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.612369    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"]
Sep 18 15:42:33.615924 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.615904    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.618595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.618576    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 15:42:33.619660 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.619645    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g427q\""
Sep 18 15:42:33.619660 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.619652    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 15:42:33.624675 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.624653    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"]
Sep 18 15:42:33.711643 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.711609    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"]
Sep 18 15:42:33.714950 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.714929    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:33.724152 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.724129    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"]
Sep 18 15:42:33.778514 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.778475    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.778705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.778519    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.778705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.778608    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr9zj\" (UniqueName: \"kubernetes.io/projected/6c3e593f-f700-4337-a5f9-39ebaa99e14e-kube-api-access-wr9zj\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.813499 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.813466    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"]
Sep 18 15:42:33.816814 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.816799    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:33.825949 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.825930    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"]
Sep 18 15:42:33.879530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.879455    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h52mx\" (UniqueName: \"kubernetes.io/projected/be17cdc4-547e-479f-a454-e0442b823d5d-kube-api-access-h52mx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:33.879530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.879491    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.879530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.879512    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.879734 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.879558    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:33.879734 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.879578    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:33.879734 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.879595    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr9zj\" (UniqueName: \"kubernetes.io/projected/6c3e593f-f700-4337-a5f9-39ebaa99e14e-kube-api-access-wr9zj\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.879884 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.879865    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.879935 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.879884    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.889344 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.889324    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr9zj\" (UniqueName: \"kubernetes.io/projected/6c3e593f-f700-4337-a5f9-39ebaa99e14e-kube-api-access-wr9zj\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.916908 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.916884    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"]
Sep 18 15:42:33.920314 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.920301    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:33.925750 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.925731    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:33.930898 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.930878    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"]
Sep 18 15:42:33.981341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.980579    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:33.981341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.980633    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:33.981341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.980673    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpdp6\" (UniqueName: \"kubernetes.io/projected/426a5961-75c9-4494-8bae-fa8311c19f83-kube-api-access-cpdp6\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:33.981341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.980723    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:33.981341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.980772    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h52mx\" (UniqueName: \"kubernetes.io/projected/be17cdc4-547e-479f-a454-e0442b823d5d-kube-api-access-h52mx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:33.981341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.980886    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:33.981341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.981266    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:33.981809 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.981473    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:33.993002 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:33.992975    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h52mx\" (UniqueName: \"kubernetes.io/projected/be17cdc4-547e-479f-a454-e0442b823d5d-kube-api-access-h52mx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:34.024892 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.024870    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:34.082455 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.082373    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:34.082455 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.082424    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpdp6\" (UniqueName: \"kubernetes.io/projected/426a5961-75c9-4494-8bae-fa8311c19f83-kube-api-access-cpdp6\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:34.082612 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.082471    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:34.082612 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.082523    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.082612 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.082560    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjwqs\" (UniqueName: \"kubernetes.io/projected/06818fff-5abe-43d1-aae0-6b414794a589-kube-api-access-wjwqs\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.082765 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.082664    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.082895 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.082872    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:34.082943 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.082889    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:34.091693 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.091668    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpdp6\" (UniqueName: \"kubernetes.io/projected/426a5961-75c9-4494-8bae-fa8311c19f83-kube-api-access-cpdp6\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:34.126672 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.126642    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:34.183766 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.183736    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.183914 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.183793    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjwqs\" (UniqueName: \"kubernetes.io/projected/06818fff-5abe-43d1-aae0-6b414794a589-kube-api-access-wjwqs\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.183914 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.183868    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.184358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.184326    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.184683 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.184592    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.195613 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.195593    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjwqs\" (UniqueName: \"kubernetes.io/projected/06818fff-5abe-43d1-aae0-6b414794a589-kube-api-access-wjwqs\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.203270 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.203246    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"]
Sep 18 15:42:34.205939 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:42:34.205915    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c3e593f_f700_4337_a5f9_39ebaa99e14e.slice/crio-f5679a737f86e05819cdb3f1cb092c616d54393b6d1ecdd7d414c1a125b825f7 WatchSource:0}: Error finding container f5679a737f86e05819cdb3f1cb092c616d54393b6d1ecdd7d414c1a125b825f7: Status 404 returned error can't find the container with id f5679a737f86e05819cdb3f1cb092c616d54393b6d1ecdd7d414c1a125b825f7
Sep 18 15:42:34.230228 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.230203    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:34.302254 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.302225    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"]
Sep 18 15:42:34.304261 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:42:34.304234    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe17cdc4_547e_479f_a454_e0442b823d5d.slice/crio-431ac2f4378dad710f1a22ebf4acc2ab13e8cbf6bb155742a40861e80e26c1bc WatchSource:0}: Error finding container 431ac2f4378dad710f1a22ebf4acc2ab13e8cbf6bb155742a40861e80e26c1bc: Status 404 returned error can't find the container with id 431ac2f4378dad710f1a22ebf4acc2ab13e8cbf6bb155742a40861e80e26c1bc
Sep 18 15:42:34.398786 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.398763    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"]
Sep 18 15:42:34.401515 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:42:34.401487    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426a5961_75c9_4494_8bae_fa8311c19f83.slice/crio-3b4ceea2a537cbfa49bab3f4afb18efb64e9fbefa430633538e38679c9732861 WatchSource:0}: Error finding container 3b4ceea2a537cbfa49bab3f4afb18efb64e9fbefa430633538e38679c9732861: Status 404 returned error can't find the container with id 3b4ceea2a537cbfa49bab3f4afb18efb64e9fbefa430633538e38679c9732861
Sep 18 15:42:34.508754 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.508732    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"]
Sep 18 15:42:34.510138 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:42:34.510111    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06818fff_5abe_43d1_aae0_6b414794a589.slice/crio-98ba9af5a027717906a097f883c1755ec2b16420051debee03e716cdc6972cf9 WatchSource:0}: Error finding container 98ba9af5a027717906a097f883c1755ec2b16420051debee03e716cdc6972cf9: Status 404 returned error can't find the container with id 98ba9af5a027717906a097f883c1755ec2b16420051debee03e716cdc6972cf9
Sep 18 15:42:34.548687 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.548657    2585 generic.go:356] "Generic (PLEG): container finished" podID="be17cdc4-547e-479f-a454-e0442b823d5d" containerID="92cfa37e9026043c8f8efea9c2209d4686d72beb6259a920660ca810149c087f" exitCode=0
Sep 18 15:42:34.548793 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.548739    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv" event={"ID":"be17cdc4-547e-479f-a454-e0442b823d5d","Type":"ContainerDied","Data":"92cfa37e9026043c8f8efea9c2209d4686d72beb6259a920660ca810149c087f"}
Sep 18 15:42:34.548872 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.548794    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv" event={"ID":"be17cdc4-547e-479f-a454-e0442b823d5d","Type":"ContainerStarted","Data":"431ac2f4378dad710f1a22ebf4acc2ab13e8cbf6bb155742a40861e80e26c1bc"}
Sep 18 15:42:34.550394 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.550349    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts" event={"ID":"06818fff-5abe-43d1-aae0-6b414794a589","Type":"ContainerStarted","Data":"98ba9af5a027717906a097f883c1755ec2b16420051debee03e716cdc6972cf9"}
Sep 18 15:42:34.552787 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.552761    2585 generic.go:356] "Generic (PLEG): container finished" podID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerID="0ec07b0effd0357b0edc0ff8a088532aecd25f03ea4d25568925a6de71eb2e23" exitCode=0
Sep 18 15:42:34.552941 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.552917    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2" event={"ID":"6c3e593f-f700-4337-a5f9-39ebaa99e14e","Type":"ContainerDied","Data":"0ec07b0effd0357b0edc0ff8a088532aecd25f03ea4d25568925a6de71eb2e23"}
Sep 18 15:42:34.553056 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.552962    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2" event={"ID":"6c3e593f-f700-4337-a5f9-39ebaa99e14e","Type":"ContainerStarted","Data":"f5679a737f86e05819cdb3f1cb092c616d54393b6d1ecdd7d414c1a125b825f7"}
Sep 18 15:42:34.554691 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.554665    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m" event={"ID":"426a5961-75c9-4494-8bae-fa8311c19f83","Type":"ContainerStarted","Data":"59b1e209c25d5ebb7c6632c51ab9e08261bc26f2b07420236f02d698fff66346"}
Sep 18 15:42:34.554763 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:34.554709    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m" event={"ID":"426a5961-75c9-4494-8bae-fa8311c19f83","Type":"ContainerStarted","Data":"3b4ceea2a537cbfa49bab3f4afb18efb64e9fbefa430633538e38679c9732861"}
Sep 18 15:42:35.565214 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:35.565185    2585 generic.go:356] "Generic (PLEG): container finished" podID="426a5961-75c9-4494-8bae-fa8311c19f83" containerID="59b1e209c25d5ebb7c6632c51ab9e08261bc26f2b07420236f02d698fff66346" exitCode=0
Sep 18 15:42:35.565553 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:35.565270    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m" event={"ID":"426a5961-75c9-4494-8bae-fa8311c19f83","Type":"ContainerDied","Data":"59b1e209c25d5ebb7c6632c51ab9e08261bc26f2b07420236f02d698fff66346"}
Sep 18 15:42:35.567298 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:35.567276    2585 generic.go:356] "Generic (PLEG): container finished" podID="be17cdc4-547e-479f-a454-e0442b823d5d" containerID="08c002ba98387c87f904e56eea531d8a628f62847322530a0551c99a9b6e37c2" exitCode=0
Sep 18 15:42:35.567400 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:35.567357    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv" event={"ID":"be17cdc4-547e-479f-a454-e0442b823d5d","Type":"ContainerDied","Data":"08c002ba98387c87f904e56eea531d8a628f62847322530a0551c99a9b6e37c2"}
Sep 18 15:42:35.569193 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:35.569170    2585 generic.go:356] "Generic (PLEG): container finished" podID="06818fff-5abe-43d1-aae0-6b414794a589" containerID="5d83e4791880809bcdf48a1eb6b1febeee7aba31b6020ae42bfb7d98dfefcd1f" exitCode=0
Sep 18 15:42:35.569320 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:35.569257    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts" event={"ID":"06818fff-5abe-43d1-aae0-6b414794a589","Type":"ContainerDied","Data":"5d83e4791880809bcdf48a1eb6b1febeee7aba31b6020ae42bfb7d98dfefcd1f"}
Sep 18 15:42:35.571355 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:35.571332    2585 generic.go:356] "Generic (PLEG): container finished" podID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerID="4fffdc7afdcdc1cc6cb6aa0a611ea60090ad265941f3b0331ae83ed2247e20a7" exitCode=0
Sep 18 15:42:35.571453 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:35.571414    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2" event={"ID":"6c3e593f-f700-4337-a5f9-39ebaa99e14e","Type":"ContainerDied","Data":"4fffdc7afdcdc1cc6cb6aa0a611ea60090ad265941f3b0331ae83ed2247e20a7"}
Sep 18 15:42:35.766407 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:42:35.766381    2585 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe17cdc4_547e_479f_a454_e0442b823d5d.slice/crio-6a35d375cc6805ec48cf07d7ada2591b2141f4e5c8f3b046e46283b86f1f6c78.scope\": RecentStats: unable to find data in memory cache]"
Sep 18 15:42:36.577876 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:36.577833    2585 generic.go:356] "Generic (PLEG): container finished" podID="06818fff-5abe-43d1-aae0-6b414794a589" containerID="03030e21f972429a1edca63cc8b8c5ea25b42176e2536a7d575884d04eef3f3f" exitCode=0
Sep 18 15:42:36.578166 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:36.577873    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts" event={"ID":"06818fff-5abe-43d1-aae0-6b414794a589","Type":"ContainerDied","Data":"03030e21f972429a1edca63cc8b8c5ea25b42176e2536a7d575884d04eef3f3f"}
Sep 18 15:42:36.580294 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:36.580275    2585 generic.go:356] "Generic (PLEG): container finished" podID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerID="7f396f2186fc6967cb2aa8e2d796f3d7a9be716e52dc8ae79c107af1ed294181" exitCode=0
Sep 18 15:42:36.580390 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:36.580304    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2" event={"ID":"6c3e593f-f700-4337-a5f9-39ebaa99e14e","Type":"ContainerDied","Data":"7f396f2186fc6967cb2aa8e2d796f3d7a9be716e52dc8ae79c107af1ed294181"}
Sep 18 15:42:36.582219 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:36.582201    2585 generic.go:356] "Generic (PLEG): container finished" podID="426a5961-75c9-4494-8bae-fa8311c19f83" containerID="57981609eb1b76b210f30ed59aec983ba9f9cd1eeb91e3abedb4ce781716249f" exitCode=0
Sep 18 15:42:36.582326 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:36.582283    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m" event={"ID":"426a5961-75c9-4494-8bae-fa8311c19f83","Type":"ContainerDied","Data":"57981609eb1b76b210f30ed59aec983ba9f9cd1eeb91e3abedb4ce781716249f"}
Sep 18 15:42:36.584644 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:36.584622    2585 generic.go:356] "Generic (PLEG): container finished" podID="be17cdc4-547e-479f-a454-e0442b823d5d" containerID="6a35d375cc6805ec48cf07d7ada2591b2141f4e5c8f3b046e46283b86f1f6c78" exitCode=0
Sep 18 15:42:36.584736 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:36.584656    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv" event={"ID":"be17cdc4-547e-479f-a454-e0442b823d5d","Type":"ContainerDied","Data":"6a35d375cc6805ec48cf07d7ada2591b2141f4e5c8f3b046e46283b86f1f6c78"}
Sep 18 15:42:37.591087 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.591052    2585 generic.go:356] "Generic (PLEG): container finished" podID="06818fff-5abe-43d1-aae0-6b414794a589" containerID="fe7c25e15ce731b01e690618ae40b5694b92a32319dec339abff46889eedca07" exitCode=0
Sep 18 15:42:37.591457 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.591131    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts" event={"ID":"06818fff-5abe-43d1-aae0-6b414794a589","Type":"ContainerDied","Data":"fe7c25e15ce731b01e690618ae40b5694b92a32319dec339abff46889eedca07"}
Sep 18 15:42:37.593100 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.593079    2585 generic.go:356] "Generic (PLEG): container finished" podID="426a5961-75c9-4494-8bae-fa8311c19f83" containerID="438890c989f02e6d4a6eb4e49e0b8fe1ff9920780c8788004e1dcd21aa2370e3" exitCode=0
Sep 18 15:42:37.593205 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.593160    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m" event={"ID":"426a5961-75c9-4494-8bae-fa8311c19f83","Type":"ContainerDied","Data":"438890c989f02e6d4a6eb4e49e0b8fe1ff9920780c8788004e1dcd21aa2370e3"}
Sep 18 15:42:37.726075 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.726051    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:37.754469 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.754447    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:37.820414 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.820385    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr9zj\" (UniqueName: \"kubernetes.io/projected/6c3e593f-f700-4337-a5f9-39ebaa99e14e-kube-api-access-wr9zj\") pod \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") "
Sep 18 15:42:37.820553 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.820463    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-bundle\") pod \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") "
Sep 18 15:42:37.820633 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.820615    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-util\") pod \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\" (UID: \"6c3e593f-f700-4337-a5f9-39ebaa99e14e\") "
Sep 18 15:42:37.820999 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.820974    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-bundle" (OuterVolumeSpecName: "bundle") pod "6c3e593f-f700-4337-a5f9-39ebaa99e14e" (UID: "6c3e593f-f700-4337-a5f9-39ebaa99e14e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:42:37.822416 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.822393    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c3e593f-f700-4337-a5f9-39ebaa99e14e-kube-api-access-wr9zj" (OuterVolumeSpecName: "kube-api-access-wr9zj") pod "6c3e593f-f700-4337-a5f9-39ebaa99e14e" (UID: "6c3e593f-f700-4337-a5f9-39ebaa99e14e"). InnerVolumeSpecName "kube-api-access-wr9zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:42:37.826254 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.826214    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-util" (OuterVolumeSpecName: "util") pod "6c3e593f-f700-4337-a5f9-39ebaa99e14e" (UID: "6c3e593f-f700-4337-a5f9-39ebaa99e14e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:42:37.922022 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.921992    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h52mx\" (UniqueName: \"kubernetes.io/projected/be17cdc4-547e-479f-a454-e0442b823d5d-kube-api-access-h52mx\") pod \"be17cdc4-547e-479f-a454-e0442b823d5d\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") "
Sep 18 15:42:37.922145 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.922040    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-util\") pod \"be17cdc4-547e-479f-a454-e0442b823d5d\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") "
Sep 18 15:42:37.922145 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.922102    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-bundle\") pod \"be17cdc4-547e-479f-a454-e0442b823d5d\" (UID: \"be17cdc4-547e-479f-a454-e0442b823d5d\") "
Sep 18 15:42:37.922283 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.922265    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:37.922283 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.922279    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c3e593f-f700-4337-a5f9-39ebaa99e14e-util\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:37.922386 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.922287    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr9zj\" (UniqueName: \"kubernetes.io/projected/6c3e593f-f700-4337-a5f9-39ebaa99e14e-kube-api-access-wr9zj\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:37.922611 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.922578    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-bundle" (OuterVolumeSpecName: "bundle") pod "be17cdc4-547e-479f-a454-e0442b823d5d" (UID: "be17cdc4-547e-479f-a454-e0442b823d5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:42:37.923999 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.923977    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be17cdc4-547e-479f-a454-e0442b823d5d-kube-api-access-h52mx" (OuterVolumeSpecName: "kube-api-access-h52mx") pod "be17cdc4-547e-479f-a454-e0442b823d5d" (UID: "be17cdc4-547e-479f-a454-e0442b823d5d"). InnerVolumeSpecName "kube-api-access-h52mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:42:37.927530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:37.927507    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-util" (OuterVolumeSpecName: "util") pod "be17cdc4-547e-479f-a454-e0442b823d5d" (UID: "be17cdc4-547e-479f-a454-e0442b823d5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:42:38.023042 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.022997    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h52mx\" (UniqueName: \"kubernetes.io/projected/be17cdc4-547e-479f-a454-e0442b823d5d-kube-api-access-h52mx\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:38.023042 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.023036    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-util\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:38.023042 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.023048    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be17cdc4-547e-479f-a454-e0442b823d5d-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:38.599701 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.599671    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv"
Sep 18 15:42:38.600130 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.599694    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c4pzfv" event={"ID":"be17cdc4-547e-479f-a454-e0442b823d5d","Type":"ContainerDied","Data":"431ac2f4378dad710f1a22ebf4acc2ab13e8cbf6bb155742a40861e80e26c1bc"}
Sep 18 15:42:38.600130 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.599749    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="431ac2f4378dad710f1a22ebf4acc2ab13e8cbf6bb155742a40861e80e26c1bc"
Sep 18 15:42:38.601770 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.601750    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2"
Sep 18 15:42:38.601907 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.601778    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w62w2" event={"ID":"6c3e593f-f700-4337-a5f9-39ebaa99e14e","Type":"ContainerDied","Data":"f5679a737f86e05819cdb3f1cb092c616d54393b6d1ecdd7d414c1a125b825f7"}
Sep 18 15:42:38.601907 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.601825    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5679a737f86e05819cdb3f1cb092c616d54393b6d1ecdd7d414c1a125b825f7"
Sep 18 15:42:38.744190 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.744167    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:38.747498 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.747479    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:38.829625 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.829596    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-util\") pod \"06818fff-5abe-43d1-aae0-6b414794a589\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") "
Sep 18 15:42:38.829763 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.829631    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-bundle\") pod \"06818fff-5abe-43d1-aae0-6b414794a589\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") "
Sep 18 15:42:38.829763 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.829662    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjwqs\" (UniqueName: \"kubernetes.io/projected/06818fff-5abe-43d1-aae0-6b414794a589-kube-api-access-wjwqs\") pod \"06818fff-5abe-43d1-aae0-6b414794a589\" (UID: \"06818fff-5abe-43d1-aae0-6b414794a589\") "
Sep 18 15:42:38.830307 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.830285    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-bundle" (OuterVolumeSpecName: "bundle") pod "06818fff-5abe-43d1-aae0-6b414794a589" (UID: "06818fff-5abe-43d1-aae0-6b414794a589"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:42:38.831663 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.831646    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06818fff-5abe-43d1-aae0-6b414794a589-kube-api-access-wjwqs" (OuterVolumeSpecName: "kube-api-access-wjwqs") pod "06818fff-5abe-43d1-aae0-6b414794a589" (UID: "06818fff-5abe-43d1-aae0-6b414794a589"). InnerVolumeSpecName "kube-api-access-wjwqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:42:38.834296 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.834277    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-util" (OuterVolumeSpecName: "util") pod "06818fff-5abe-43d1-aae0-6b414794a589" (UID: "06818fff-5abe-43d1-aae0-6b414794a589"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:42:38.930570 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.930542    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-util\") pod \"426a5961-75c9-4494-8bae-fa8311c19f83\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") "
Sep 18 15:42:38.930658 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.930603    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-bundle\") pod \"426a5961-75c9-4494-8bae-fa8311c19f83\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") "
Sep 18 15:42:38.930722 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.930708    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpdp6\" (UniqueName: \"kubernetes.io/projected/426a5961-75c9-4494-8bae-fa8311c19f83-kube-api-access-cpdp6\") pod \"426a5961-75c9-4494-8bae-fa8311c19f83\" (UID: \"426a5961-75c9-4494-8bae-fa8311c19f83\") "
Sep 18 15:42:38.930963 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.930948    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-util\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:38.931016 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.930969    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06818fff-5abe-43d1-aae0-6b414794a589-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:38.931016 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.930985    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjwqs\" (UniqueName: \"kubernetes.io/projected/06818fff-5abe-43d1-aae0-6b414794a589-kube-api-access-wjwqs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:38.931016 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.931000    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-bundle" (OuterVolumeSpecName: "bundle") pod "426a5961-75c9-4494-8bae-fa8311c19f83" (UID: "426a5961-75c9-4494-8bae-fa8311c19f83"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:42:38.932658 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.932633    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/426a5961-75c9-4494-8bae-fa8311c19f83-kube-api-access-cpdp6" (OuterVolumeSpecName: "kube-api-access-cpdp6") pod "426a5961-75c9-4494-8bae-fa8311c19f83" (UID: "426a5961-75c9-4494-8bae-fa8311c19f83"). InnerVolumeSpecName "kube-api-access-cpdp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:42:38.936261 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:38.936239    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-util" (OuterVolumeSpecName: "util") pod "426a5961-75c9-4494-8bae-fa8311c19f83" (UID: "426a5961-75c9-4494-8bae-fa8311c19f83"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:42:39.031803 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:39.031777    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-util\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:39.031803 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:39.031801    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/426a5961-75c9-4494-8bae-fa8311c19f83-bundle\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:39.031975 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:39.031810    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpdp6\" (UniqueName: \"kubernetes.io/projected/426a5961-75c9-4494-8bae-fa8311c19f83-kube-api-access-cpdp6\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:42:39.608390 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:39.608360    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts"
Sep 18 15:42:39.608884 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:39.608393    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hsts" event={"ID":"06818fff-5abe-43d1-aae0-6b414794a589","Type":"ContainerDied","Data":"98ba9af5a027717906a097f883c1755ec2b16420051debee03e716cdc6972cf9"}
Sep 18 15:42:39.608884 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:39.608453    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98ba9af5a027717906a097f883c1755ec2b16420051debee03e716cdc6972cf9"
Sep 18 15:42:39.610313 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:39.610289    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m" event={"ID":"426a5961-75c9-4494-8bae-fa8311c19f83","Type":"ContainerDied","Data":"3b4ceea2a537cbfa49bab3f4afb18efb64e9fbefa430633538e38679c9732861"}
Sep 18 15:42:39.610496 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:39.610473    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b4ceea2a537cbfa49bab3f4afb18efb64e9fbefa430633538e38679c9732861"
Sep 18 15:42:39.610496 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:39.610354    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79xd4m"
Sep 18 15:42:44.560596 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560562    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"]
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560889    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06818fff-5abe-43d1-aae0-6b414794a589" containerName="util"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560900    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="06818fff-5abe-43d1-aae0-6b414794a589" containerName="util"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560907    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerName="util"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560913    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerName="util"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560918    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerName="pull"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560923    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerName="pull"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560928    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06818fff-5abe-43d1-aae0-6b414794a589" containerName="pull"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560933    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="06818fff-5abe-43d1-aae0-6b414794a589" containerName="pull"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560940    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426a5961-75c9-4494-8bae-fa8311c19f83" containerName="extract"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560945    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="426a5961-75c9-4494-8bae-fa8311c19f83" containerName="extract"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560953    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426a5961-75c9-4494-8bae-fa8311c19f83" containerName="util"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560958    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="426a5961-75c9-4494-8bae-fa8311c19f83" containerName="util"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560966    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be17cdc4-547e-479f-a454-e0442b823d5d" containerName="pull"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560971    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="be17cdc4-547e-479f-a454-e0442b823d5d" containerName="pull"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560977    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerName="extract"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560982    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerName="extract"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560990    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be17cdc4-547e-479f-a454-e0442b823d5d" containerName="extract"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.560996    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="be17cdc4-547e-479f-a454-e0442b823d5d" containerName="extract"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561004    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be17cdc4-547e-479f-a454-e0442b823d5d" containerName="util"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561010    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="be17cdc4-547e-479f-a454-e0442b823d5d" containerName="util"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561015    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06818fff-5abe-43d1-aae0-6b414794a589" containerName="extract"
Sep 18 15:42:44.561014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561019    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="06818fff-5abe-43d1-aae0-6b414794a589" containerName="extract"
Sep 18 15:42:44.561685 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561027    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426a5961-75c9-4494-8bae-fa8311c19f83" containerName="pull"
Sep 18 15:42:44.561685 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561032    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="426a5961-75c9-4494-8bae-fa8311c19f83" containerName="pull"
Sep 18 15:42:44.561685 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561084    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="426a5961-75c9-4494-8bae-fa8311c19f83" containerName="extract"
Sep 18 15:42:44.561685 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561093    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="be17cdc4-547e-479f-a454-e0442b823d5d" containerName="extract"
Sep 18 15:42:44.561685 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561099    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="06818fff-5abe-43d1-aae0-6b414794a589" containerName="extract"
Sep 18 15:42:44.561685 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.561105    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c3e593f-f700-4337-a5f9-39ebaa99e14e" containerName="extract"
Sep 18 15:42:44.565182 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.565166    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:42:44.568960 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.568935    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 15:42:44.569065 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.569006    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 15:42:44.570137 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.570123    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5g9b5\""
Sep 18 15:42:44.593397 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.593377    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"]
Sep 18 15:42:44.680426 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.680405    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4t5h\" (UniqueName: \"kubernetes.io/projected/3db9bb36-9f90-45c1-bcc4-a700c1939953-kube-api-access-p4t5h\") pod \"limitador-operator-controller-manager-74495c69f8-np6fn\" (UID: \"3db9bb36-9f90-45c1-bcc4-a700c1939953\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:42:44.781701 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.781669    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4t5h\" (UniqueName: \"kubernetes.io/projected/3db9bb36-9f90-45c1-bcc4-a700c1939953-kube-api-access-p4t5h\") pod \"limitador-operator-controller-manager-74495c69f8-np6fn\" (UID: \"3db9bb36-9f90-45c1-bcc4-a700c1939953\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:42:44.791289 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.791260    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4t5h\" (UniqueName: \"kubernetes.io/projected/3db9bb36-9f90-45c1-bcc4-a700c1939953-kube-api-access-p4t5h\") pod \"limitador-operator-controller-manager-74495c69f8-np6fn\" (UID: \"3db9bb36-9f90-45c1-bcc4-a700c1939953\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:42:44.875446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:44.875358    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:42:45.156012 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:45.155821    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"]
Sep 18 15:42:45.158718 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:42:45.158691    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db9bb36_9f90_45c1_bcc4_a700c1939953.slice/crio-6c55209e2f6852082e3869a36170b7504b2f2d75461a0c7aadc006f2ac786789 WatchSource:0}: Error finding container 6c55209e2f6852082e3869a36170b7504b2f2d75461a0c7aadc006f2ac786789: Status 404 returned error can't find the container with id 6c55209e2f6852082e3869a36170b7504b2f2d75461a0c7aadc006f2ac786789
Sep 18 15:42:45.641102 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:45.641065    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn" event={"ID":"3db9bb36-9f90-45c1-bcc4-a700c1939953","Type":"ContainerStarted","Data":"6c55209e2f6852082e3869a36170b7504b2f2d75461a0c7aadc006f2ac786789"}
Sep 18 15:42:47.660272 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:47.660237    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn" event={"ID":"3db9bb36-9f90-45c1-bcc4-a700c1939953","Type":"ContainerStarted","Data":"bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160"}
Sep 18 15:42:47.660661 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:47.660308    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:42:47.683865 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:47.683803    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn" podStartSLOduration=1.408108798 podStartE2EDuration="3.683789149s" totalImagesPullingTime="2.275680351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:42:47.681768953 +0000 UTC m=+492.340561613" watchObservedRunningTime="2026-09-18 15:42:47.683789149 +0000 UTC m=+492.342581809"
Sep 18 15:42:51.433110 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.433069    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"]
Sep 18 15:42:51.436658 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.436635    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:42:51.439965 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.439942    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hq9sc\""
Sep 18 15:42:51.451327 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.451305    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"]
Sep 18 15:42:51.528850 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.528803    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" (UID: \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:42:51.529022 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.528925    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78jw2\" (UniqueName: \"kubernetes.io/projected/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-kube-api-access-78jw2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" (UID: \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:42:51.630214 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.630178    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78jw2\" (UniqueName: \"kubernetes.io/projected/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-kube-api-access-78jw2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" (UID: \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:42:51.630383 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.630234    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" (UID: \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:42:51.630622 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.630600    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" (UID: \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:42:51.640394 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.640361    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78jw2\" (UniqueName: \"kubernetes.io/projected/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-kube-api-access-78jw2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" (UID: \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:42:51.746770 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:51.746747    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:42:52.049412 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:52.049387    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"]
Sep 18 15:42:52.051238 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:42:52.051211    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod362275a1_ddc3_46c0_a9a9_67be04ad8fd4.slice/crio-6f6ce936b5e9dbb9609d37ec7ab8c8bdfea30f2a5d3e8c93b03b112a1e933725 WatchSource:0}: Error finding container 6f6ce936b5e9dbb9609d37ec7ab8c8bdfea30f2a5d3e8c93b03b112a1e933725: Status 404 returned error can't find the container with id 6f6ce936b5e9dbb9609d37ec7ab8c8bdfea30f2a5d3e8c93b03b112a1e933725
Sep 18 15:42:52.687033 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:52.686991    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s" event={"ID":"362275a1-ddc3-46c0-a9a9-67be04ad8fd4","Type":"ContainerStarted","Data":"6f6ce936b5e9dbb9609d37ec7ab8c8bdfea30f2a5d3e8c93b03b112a1e933725"}
Sep 18 15:42:56.718416 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:56.718373    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s" event={"ID":"362275a1-ddc3-46c0-a9a9-67be04ad8fd4","Type":"ContainerStarted","Data":"7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970"}
Sep 18 15:42:56.718824 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:56.718518    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:42:58.667438 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:58.667408    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:42:58.686079 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:42:58.686031    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s" podStartSLOduration=3.933848961 podStartE2EDuration="7.686015791s" totalImagesPullingTime="3.75216683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:42:56.754079049 +0000 UTC m=+501.412871708" watchObservedRunningTime="2026-09-18 15:42:58.686015791 +0000 UTC m=+503.344808451"
Sep 18 15:43:07.725812 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:07.725779    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.058408 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.058373    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"]
Sep 18 15:43:10.058818 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.058584    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" containerName="manager" containerID="cri-o://7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970" gracePeriod=2
Sep 18 15:43:10.071200 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.071172    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"]
Sep 18 15:43:10.086830 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.086805    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"]
Sep 18 15:43:10.087054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.087018    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" containerName="manager" containerID="cri-o://bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160" gracePeriod=2
Sep 18 15:43:10.088107 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.088067    2585 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\": pods \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.091037 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.090994    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.092152 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.092130    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"]
Sep 18 15:43:10.092482 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.092467    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" containerName="manager"
Sep 18 15:43:10.092559 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.092493    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" containerName="manager"
Sep 18 15:43:10.092668 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.092587    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" containerName="manager"
Sep 18 15:43:10.095606 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.095587    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:10.100552 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.100534    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"]
Sep 18 15:43:10.104091 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.104065    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.106281 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.106250    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-np6fn\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:43:10.110668 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.110643    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"]
Sep 18 15:43:10.114534 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.114510    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"]
Sep 18 15:43:10.114878 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.114861    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" containerName="manager"
Sep 18 15:43:10.114878 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.114877    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" containerName="manager"
Sep 18 15:43:10.115021 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.114944    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" containerName="manager"
Sep 18 15:43:10.117717 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.117699    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"
Sep 18 15:43:10.134194 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.134164    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-np6fn\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:43:10.141402 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.141383    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"]
Sep 18 15:43:10.163980 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.163950    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.178291 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.178265    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84a170f1-17e8-4f95-b4ef-608ccb746bdb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-g885b\" (UID: \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:10.178399 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.178308    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnjnc\" (UniqueName: \"kubernetes.io/projected/5720ae4e-58b4-445c-ba0b-dafef88e6c0e-kube-api-access-hnjnc\") pod \"limitador-operator-controller-manager-74495c69f8-mwxbq\" (UID: \"5720ae4e-58b4-445c-ba0b-dafef88e6c0e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"
Sep 18 15:43:10.178399 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.178339    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2k24\" (UniqueName: \"kubernetes.io/projected/84a170f1-17e8-4f95-b4ef-608ccb746bdb-kube-api-access-h2k24\") pod \"kuadrant-operator-controller-manager-76f8f686bf-g885b\" (UID: \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:10.279434 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.279407    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84a170f1-17e8-4f95-b4ef-608ccb746bdb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-g885b\" (UID: \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:10.279588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.279445    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnjnc\" (UniqueName: \"kubernetes.io/projected/5720ae4e-58b4-445c-ba0b-dafef88e6c0e-kube-api-access-hnjnc\") pod \"limitador-operator-controller-manager-74495c69f8-mwxbq\" (UID: \"5720ae4e-58b4-445c-ba0b-dafef88e6c0e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"
Sep 18 15:43:10.279588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.279548    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2k24\" (UniqueName: \"kubernetes.io/projected/84a170f1-17e8-4f95-b4ef-608ccb746bdb-kube-api-access-h2k24\") pod \"kuadrant-operator-controller-manager-76f8f686bf-g885b\" (UID: \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:10.279797 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.279779    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84a170f1-17e8-4f95-b4ef-608ccb746bdb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-g885b\" (UID: \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:10.288738 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.288709    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2k24\" (UniqueName: \"kubernetes.io/projected/84a170f1-17e8-4f95-b4ef-608ccb746bdb-kube-api-access-h2k24\") pod \"kuadrant-operator-controller-manager-76f8f686bf-g885b\" (UID: \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:10.288998 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.288975    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnjnc\" (UniqueName: \"kubernetes.io/projected/5720ae4e-58b4-445c-ba0b-dafef88e6c0e-kube-api-access-hnjnc\") pod \"limitador-operator-controller-manager-74495c69f8-mwxbq\" (UID: \"5720ae4e-58b4-445c-ba0b-dafef88e6c0e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"
Sep 18 15:43:10.315860 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.315794    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.318589 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.318561    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.320802 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.320777    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-np6fn\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:43:10.328474 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.328458    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:43:10.330909 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.330882    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.333463 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.333436    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-np6fn\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:43:10.380849 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.380819    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-extensions-socket-volume\") pod \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\" (UID: \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\") "
Sep 18 15:43:10.380990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.380907    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4t5h\" (UniqueName: \"kubernetes.io/projected/3db9bb36-9f90-45c1-bcc4-a700c1939953-kube-api-access-p4t5h\") pod \"3db9bb36-9f90-45c1-bcc4-a700c1939953\" (UID: \"3db9bb36-9f90-45c1-bcc4-a700c1939953\") "
Sep 18 15:43:10.380990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.380953    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78jw2\" (UniqueName: \"kubernetes.io/projected/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-kube-api-access-78jw2\") pod \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\" (UID: \"362275a1-ddc3-46c0-a9a9-67be04ad8fd4\") "
Sep 18 15:43:10.381305 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.381281    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "362275a1-ddc3-46c0-a9a9-67be04ad8fd4" (UID: "362275a1-ddc3-46c0-a9a9-67be04ad8fd4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:43:10.382987 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.382966    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db9bb36-9f90-45c1-bcc4-a700c1939953-kube-api-access-p4t5h" (OuterVolumeSpecName: "kube-api-access-p4t5h") pod "3db9bb36-9f90-45c1-bcc4-a700c1939953" (UID: "3db9bb36-9f90-45c1-bcc4-a700c1939953"). InnerVolumeSpecName "kube-api-access-p4t5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:43:10.383044 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.382995    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-kube-api-access-78jw2" (OuterVolumeSpecName: "kube-api-access-78jw2") pod "362275a1-ddc3-46c0-a9a9-67be04ad8fd4" (UID: "362275a1-ddc3-46c0-a9a9-67be04ad8fd4"). InnerVolumeSpecName "kube-api-access-78jw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:43:10.482361 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.482337    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-extensions-socket-volume\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:43:10.482361 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.482359    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4t5h\" (UniqueName: \"kubernetes.io/projected/3db9bb36-9f90-45c1-bcc4-a700c1939953-kube-api-access-p4t5h\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:43:10.482510 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.482368    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78jw2\" (UniqueName: \"kubernetes.io/projected/362275a1-ddc3-46c0-a9a9-67be04ad8fd4-kube-api-access-78jw2\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:43:10.500280 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.500257    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:10.505229 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.505213    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"
Sep 18 15:43:10.791001 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.790971    2585 generic.go:356] "Generic (PLEG): container finished" podID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" containerID="7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970" exitCode=0
Sep 18 15:43:10.791166 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.791015    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.791166 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.791070    2585 scope.go:117] "RemoveContainer" containerID="7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970"
Sep 18 15:43:10.792660 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.792638    2585 generic.go:356] "Generic (PLEG): container finished" podID="3db9bb36-9f90-45c1-bcc4-a700c1939953" containerID="bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160" exitCode=0
Sep 18 15:43:10.792752 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.792680    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:43:10.798478 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.798439    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-np6fn\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:43:10.800142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.800123    2585 scope.go:117] "RemoveContainer" containerID="7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970"
Sep 18 15:43:10.800463 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.800446    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970\": container with ID starting with 7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970 not found: ID does not exist" containerID="7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970"
Sep 18 15:43:10.800540 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.800469    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970"} err="failed to get container status \"7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970\": rpc error: code = NotFound desc = could not find container \"7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970\": container with ID starting with 7989e3157aa46eda68e0465d4133fd792b219c5460fe7a86d74733ffe127d970 not found: ID does not exist"
Sep 18 15:43:10.800540 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.800489    2585 scope.go:117] "RemoveContainer" containerID="bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160"
Sep 18 15:43:10.801969 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.801947    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.805063 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.805030    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-np6fn\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:43:10.809592 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.809473    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.809659 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.809631    2585 scope.go:117] "RemoveContainer" containerID="bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160"
Sep 18 15:43:10.809940 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.809922    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160\": container with ID starting with bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160 not found: ID does not exist" containerID="bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160"
Sep 18 15:43:10.810010 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.809951    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160"} err="failed to get container status \"bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160\": rpc error: code = NotFound desc = could not find container \"bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160\": container with ID starting with bf8aaf873fadab5916d0f8ea253b7cbe9bd3c54bc8f31e28aff6e3a00580e160 not found: ID does not exist"
Sep 18 15:43:10.811000 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.810977    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"]
Sep 18 15:43:10.811075 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:43:10.811020    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a170f1_17e8_4f95_b4ef_608ccb746bdb.slice/crio-4bc7459f569bf4fa9e85976359a40ee018a05e246fd3fbd28633037edc3f4aee WatchSource:0}: Error finding container 4bc7459f569bf4fa9e85976359a40ee018a05e246fd3fbd28633037edc3f4aee: Status 404 returned error can't find the container with id 4bc7459f569bf4fa9e85976359a40ee018a05e246fd3fbd28633037edc3f4aee
Sep 18 15:43:10.812208 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.812187    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-jpf4s\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jpf4s"
Sep 18 15:43:10.821817 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:10.821795    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-np6fn\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-np6fn"
Sep 18 15:43:10.831517 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:10.831493    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"]
Sep 18 15:43:10.832639 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:43:10.832618    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5720ae4e_58b4_445c_ba0b_dafef88e6c0e.slice/crio-c1892d0f8339fe278892a85bfcf56c6fecef0c5a3d8a738312690bad3fdee8c5 WatchSource:0}: Error finding container c1892d0f8339fe278892a85bfcf56c6fecef0c5a3d8a738312690bad3fdee8c5: Status 404 returned error can't find the container with id c1892d0f8339fe278892a85bfcf56c6fecef0c5a3d8a738312690bad3fdee8c5
Sep 18 15:43:11.527563 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.527520    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="362275a1-ddc3-46c0-a9a9-67be04ad8fd4" path="/var/lib/kubelet/pods/362275a1-ddc3-46c0-a9a9-67be04ad8fd4/volumes"
Sep 18 15:43:11.528050 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.527830    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3db9bb36-9f90-45c1-bcc4-a700c1939953" path="/var/lib/kubelet/pods/3db9bb36-9f90-45c1-bcc4-a700c1939953/volumes"
Sep 18 15:43:11.801674 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.801567    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b" event={"ID":"84a170f1-17e8-4f95-b4ef-608ccb746bdb","Type":"ContainerStarted","Data":"885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4"}
Sep 18 15:43:11.801674 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.801635    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:11.801674 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.801661    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b" event={"ID":"84a170f1-17e8-4f95-b4ef-608ccb746bdb","Type":"ContainerStarted","Data":"4bc7459f569bf4fa9e85976359a40ee018a05e246fd3fbd28633037edc3f4aee"}
Sep 18 15:43:11.803088 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.803063    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq" event={"ID":"5720ae4e-58b4-445c-ba0b-dafef88e6c0e","Type":"ContainerStarted","Data":"cf60539235319c2fbb9b306cc9e01f3cfc5b10703d97961169ba96aa58b7ff76"}
Sep 18 15:43:11.803178 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.803110    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq" event={"ID":"5720ae4e-58b4-445c-ba0b-dafef88e6c0e","Type":"ContainerStarted","Data":"c1892d0f8339fe278892a85bfcf56c6fecef0c5a3d8a738312690bad3fdee8c5"}
Sep 18 15:43:11.803178 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.803135    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"
Sep 18 15:43:11.832144 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.832103    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b" podStartSLOduration=1.832089197 podStartE2EDuration="1.832089197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:43:11.82916996 +0000 UTC m=+516.487962625" watchObservedRunningTime="2026-09-18 15:43:11.832089197 +0000 UTC m=+516.490881856"
Sep 18 15:43:11.849913 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:11.849870    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq" podStartSLOduration=1.849858154 podStartE2EDuration="1.849858154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:43:11.848040534 +0000 UTC m=+516.506833196" watchObservedRunningTime="2026-09-18 15:43:11.849858154 +0000 UTC m=+516.508650811"
Sep 18 15:43:22.810385 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:22.810309    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:22.810385 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:22.810380    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mwxbq"
Sep 18 15:43:25.839677 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:25.839640    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"]
Sep 18 15:43:25.840083 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:25.839832    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b" podUID="84a170f1-17e8-4f95-b4ef-608ccb746bdb" containerName="manager" containerID="cri-o://885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4" gracePeriod=10
Sep 18 15:43:26.092831 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.092775    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:26.186698 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.186660    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"]
Sep 18 15:43:26.187074 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.187057    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a170f1-17e8-4f95-b4ef-608ccb746bdb" containerName="manager"
Sep 18 15:43:26.187165 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.187076    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a170f1-17e8-4f95-b4ef-608ccb746bdb" containerName="manager"
Sep 18 15:43:26.187223 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.187155    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a170f1-17e8-4f95-b4ef-608ccb746bdb" containerName="manager"
Sep 18 15:43:26.190083 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.190064    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:26.215485 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.215464    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2k24\" (UniqueName: \"kubernetes.io/projected/84a170f1-17e8-4f95-b4ef-608ccb746bdb-kube-api-access-h2k24\") pod \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\" (UID: \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\") "
Sep 18 15:43:26.215599 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.215576    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84a170f1-17e8-4f95-b4ef-608ccb746bdb-extensions-socket-volume\") pod \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\" (UID: \"84a170f1-17e8-4f95-b4ef-608ccb746bdb\") "
Sep 18 15:43:26.215933 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.215912    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84a170f1-17e8-4f95-b4ef-608ccb746bdb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "84a170f1-17e8-4f95-b4ef-608ccb746bdb" (UID: "84a170f1-17e8-4f95-b4ef-608ccb746bdb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:43:26.217500 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.217478    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a170f1-17e8-4f95-b4ef-608ccb746bdb-kube-api-access-h2k24" (OuterVolumeSpecName: "kube-api-access-h2k24") pod "84a170f1-17e8-4f95-b4ef-608ccb746bdb" (UID: "84a170f1-17e8-4f95-b4ef-608ccb746bdb"). InnerVolumeSpecName "kube-api-access-h2k24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:43:26.226258 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.226233    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"]
Sep 18 15:43:26.316170 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.316139    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1061a8b-e582-4746-bbc6-1200b22fa438-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6hwrd\" (UID: \"b1061a8b-e582-4746-bbc6-1200b22fa438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:26.316335 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.316205    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hbzz\" (UniqueName: \"kubernetes.io/projected/b1061a8b-e582-4746-bbc6-1200b22fa438-kube-api-access-2hbzz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6hwrd\" (UID: \"b1061a8b-e582-4746-bbc6-1200b22fa438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:26.316335 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.316269    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2k24\" (UniqueName: \"kubernetes.io/projected/84a170f1-17e8-4f95-b4ef-608ccb746bdb-kube-api-access-h2k24\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:43:26.316335 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.316288    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84a170f1-17e8-4f95-b4ef-608ccb746bdb-extensions-socket-volume\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:43:26.416807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.416726    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1061a8b-e582-4746-bbc6-1200b22fa438-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6hwrd\" (UID: \"b1061a8b-e582-4746-bbc6-1200b22fa438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:26.416807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.416785    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hbzz\" (UniqueName: \"kubernetes.io/projected/b1061a8b-e582-4746-bbc6-1200b22fa438-kube-api-access-2hbzz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6hwrd\" (UID: \"b1061a8b-e582-4746-bbc6-1200b22fa438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:26.417211 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.417188    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1061a8b-e582-4746-bbc6-1200b22fa438-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6hwrd\" (UID: \"b1061a8b-e582-4746-bbc6-1200b22fa438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:26.433705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.433681    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hbzz\" (UniqueName: \"kubernetes.io/projected/b1061a8b-e582-4746-bbc6-1200b22fa438-kube-api-access-2hbzz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6hwrd\" (UID: \"b1061a8b-e582-4746-bbc6-1200b22fa438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:26.500274 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.500244    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:26.796177 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.796148    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"]
Sep 18 15:43:26.797433 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:43:26.797412    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1061a8b_e582_4746_bbc6_1200b22fa438.slice/crio-9154a3966fbdd0488065219b65c06f2911bef3c66e2f4c3b6869a4649a103b80 WatchSource:0}: Error finding container 9154a3966fbdd0488065219b65c06f2911bef3c66e2f4c3b6869a4649a103b80: Status 404 returned error can't find the container with id 9154a3966fbdd0488065219b65c06f2911bef3c66e2f4c3b6869a4649a103b80
Sep 18 15:43:26.883052 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.883023    2585 generic.go:356] "Generic (PLEG): container finished" podID="84a170f1-17e8-4f95-b4ef-608ccb746bdb" containerID="885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4" exitCode=0
Sep 18 15:43:26.883354 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.883069    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"
Sep 18 15:43:26.883354 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.883101    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b" event={"ID":"84a170f1-17e8-4f95-b4ef-608ccb746bdb","Type":"ContainerDied","Data":"885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4"}
Sep 18 15:43:26.883354 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.883150    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b" event={"ID":"84a170f1-17e8-4f95-b4ef-608ccb746bdb","Type":"ContainerDied","Data":"4bc7459f569bf4fa9e85976359a40ee018a05e246fd3fbd28633037edc3f4aee"}
Sep 18 15:43:26.883354 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.883172    2585 scope.go:117] "RemoveContainer" containerID="885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4"
Sep 18 15:43:26.884550 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.884528    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd" event={"ID":"b1061a8b-e582-4746-bbc6-1200b22fa438","Type":"ContainerStarted","Data":"9154a3966fbdd0488065219b65c06f2911bef3c66e2f4c3b6869a4649a103b80"}
Sep 18 15:43:26.891477 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.891457    2585 scope.go:117] "RemoveContainer" containerID="885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4"
Sep 18 15:43:26.891749 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:43:26.891732    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4\": container with ID starting with 885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4 not found: ID does not exist" containerID="885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4"
Sep 18 15:43:26.891817 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.891761    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4"} err="failed to get container status \"885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4\": rpc error: code = NotFound desc = could not find container \"885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4\": container with ID starting with 885557e98455deb96cfc80bc42c60de28e89aac046590f413029c01b357db2f4 not found: ID does not exist"
Sep 18 15:43:26.913994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.913972    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"]
Sep 18 15:43:26.919344 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:26.919324    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-g885b"]
Sep 18 15:43:27.523917 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:27.523887    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a170f1-17e8-4f95-b4ef-608ccb746bdb" path="/var/lib/kubelet/pods/84a170f1-17e8-4f95-b4ef-608ccb746bdb/volumes"
Sep 18 15:43:27.892374 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:27.892289    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd" event={"ID":"b1061a8b-e582-4746-bbc6-1200b22fa438","Type":"ContainerStarted","Data":"deb3f9d7854ef3ab06d52f84698211f742d5bbfe271bd6f50ba56580ae79587d"}
Sep 18 15:43:27.892374 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:27.892343    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:27.916175 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:27.916129    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd" podStartSLOduration=1.9161154790000001 podStartE2EDuration="1.916115479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:43:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:43:27.91346906 +0000 UTC m=+532.572261725" watchObservedRunningTime="2026-09-18 15:43:27.916115479 +0000 UTC m=+532.574908139"
Sep 18 15:43:38.899277 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:38.899243    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 15:43:42.165311 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.165274    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"]
Sep 18 15:43:42.168594 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.168573    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.171366 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.171344    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tkl6h\""
Sep 18 15:43:42.190514 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.190488    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"]
Sep 18 15:43:42.246434 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.246407    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7prjq\" (UniqueName: \"kubernetes.io/projected/9502d995-c7d8-46e9-b1d0-5342b23810d5-kube-api-access-7prjq\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.246577 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.246447    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.246577 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.246490    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.246577 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.246538    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.246682 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.246616    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.246682 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.246640    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.246796 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.246687    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.246796 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.246766    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.246886 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.246851    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9502d995-c7d8-46e9-b1d0-5342b23810d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347364 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347332    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347364 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347369    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347396    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347420    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347466    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9502d995-c7d8-46e9-b1d0-5342b23810d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347481    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7prjq\" (UniqueName: \"kubernetes.io/projected/9502d995-c7d8-46e9-b1d0-5342b23810d5-kube-api-access-7prjq\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347510    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347575    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347596    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347924 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347763    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.347924 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347877    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.348003 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347924    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.348003 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.347929    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.348274 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.348254    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9502d995-c7d8-46e9-b1d0-5342b23810d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.350025 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.349996    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.350241 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.350223    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.357942 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.357909    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7prjq\" (UniqueName: \"kubernetes.io/projected/9502d995-c7d8-46e9-b1d0-5342b23810d5-kube-api-access-7prjq\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.358030 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.357983    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9502d995-c7d8-46e9-b1d0-5342b23810d5-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6prf9\" (UID: \"9502d995-c7d8-46e9-b1d0-5342b23810d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.479074 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.478983    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:42.752975 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.752905    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"]
Sep 18 15:43:42.756442 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:43:42.756404    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9502d995_c7d8_46e9_b1d0_5342b23810d5.slice/crio-6b961dba60ce701a1cd878267f1d15fdac790605de37a134ddd2ef238a6a56fc WatchSource:0}: Error finding container 6b961dba60ce701a1cd878267f1d15fdac790605de37a134ddd2ef238a6a56fc: Status 404 returned error can't find the container with id 6b961dba60ce701a1cd878267f1d15fdac790605de37a134ddd2ef238a6a56fc
Sep 18 15:43:42.758596 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.758561    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:43:42.758681 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.758644    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:43:42.758720 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.758681    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:43:42.976640 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.976598    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" event={"ID":"9502d995-c7d8-46e9-b1d0-5342b23810d5","Type":"ContainerStarted","Data":"e091e5b16e2aebc5688261a22a354b74beb0254d90df7a269281a5e80de453bf"}
Sep 18 15:43:42.976816 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:42.976659    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" event={"ID":"9502d995-c7d8-46e9-b1d0-5342b23810d5","Type":"ContainerStarted","Data":"6b961dba60ce701a1cd878267f1d15fdac790605de37a134ddd2ef238a6a56fc"}
Sep 18 15:43:43.479638 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:43.479605    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:43.484010 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:43.483988    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:43.506740 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:43.506695    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" podStartSLOduration=1.506681302 podStartE2EDuration="1.506681302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:43:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:43:42.998154916 +0000 UTC m=+547.656947575" watchObservedRunningTime="2026-09-18 15:43:43.506681302 +0000 UTC m=+548.165474028"
Sep 18 15:43:43.987362 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:43.987331    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:43:43.988461 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:43:43.988437    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:44:37.391544 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:44:37.391514    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:44:37.392007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:44:37.391852    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:45:13.736927 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.734222    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"]
Sep 18 15:45:13.739256 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.739230    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.742250 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.742224    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 15:45:13.742250 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.742238    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 15:45:13.742400 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.742229    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 15:45:13.743279 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.743261    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hp5gf\""
Sep 18 15:45:13.743339 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.743280    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 15:45:13.749485 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.749464    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"]
Sep 18 15:45:13.833136 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.833107    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b52qv\" (UniqueName: \"kubernetes.io/projected/56059caa-5300-4bcf-90a1-e2f782a7aecc-kube-api-access-b52qv\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.833282 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.833143    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-metrics-certs\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.833282 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.833169    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56059caa-5300-4bcf-90a1-e2f782a7aecc-openshift-service-ca\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.833282 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.833271    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-maas-api-tls\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.934429 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.934399    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56059caa-5300-4bcf-90a1-e2f782a7aecc-openshift-service-ca\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.934600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.934463    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-maas-api-tls\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.934600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.934500    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b52qv\" (UniqueName: \"kubernetes.io/projected/56059caa-5300-4bcf-90a1-e2f782a7aecc-kube-api-access-b52qv\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.934600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.934522    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-metrics-certs\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.934769 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:45:13.934634    2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 18 15:45:13.934769 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:45:13.934716    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-metrics-certs podName:56059caa-5300-4bcf-90a1-e2f782a7aecc nodeName:}" failed. No retries permitted until 2026-09-18 15:45:14.434694301 +0000 UTC m=+639.093486942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-metrics-certs") pod "maas-api-59f6566bbc-ppmjg" (UID: "56059caa-5300-4bcf-90a1-e2f782a7aecc") : secret "maas-api-metrics-tls" not found
Sep 18 15:45:13.935091 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.935069    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56059caa-5300-4bcf-90a1-e2f782a7aecc-openshift-service-ca\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.937029 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.937006    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-maas-api-tls\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:13.944480 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:13.944461    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b52qv\" (UniqueName: \"kubernetes.io/projected/56059caa-5300-4bcf-90a1-e2f782a7aecc-kube-api-access-b52qv\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:14.439477 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:14.439439    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-metrics-certs\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:14.442397 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:14.442370    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-metrics-certs\") pod \"maas-api-59f6566bbc-ppmjg\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:14.651306 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:14.651270    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:14.929418 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:14.929357    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"]
Sep 18 15:45:14.934445 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:45:14.934182    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56059caa_5300_4bcf_90a1_e2f782a7aecc.slice/crio-8cfe26172d34d136cb6e83204d56d9de1d53fe5d29283569bf8c9240035c0466 WatchSource:0}: Error finding container 8cfe26172d34d136cb6e83204d56d9de1d53fe5d29283569bf8c9240035c0466: Status 404 returned error can't find the container with id 8cfe26172d34d136cb6e83204d56d9de1d53fe5d29283569bf8c9240035c0466
Sep 18 15:45:14.936641 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:14.936615    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 15:45:15.472545 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:15.472462    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg" event={"ID":"56059caa-5300-4bcf-90a1-e2f782a7aecc","Type":"ContainerStarted","Data":"8cfe26172d34d136cb6e83204d56d9de1d53fe5d29283569bf8c9240035c0466"}
Sep 18 15:45:18.492439 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:18.492402    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg" event={"ID":"56059caa-5300-4bcf-90a1-e2f782a7aecc","Type":"ContainerStarted","Data":"d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215"}
Sep 18 15:45:18.492881 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:18.492536    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:18.513207 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:18.513143    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg" podStartSLOduration=2.6004657399999997 podStartE2EDuration="5.51312211s" totalImagesPullingTime="2.91265637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:45:18.509416586 +0000 UTC m=+643.168209247" watchObservedRunningTime="2026-09-18 15:45:18.51312211 +0000 UTC m=+643.171914771"
Sep 18 15:45:23.526442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:23.526407    2585 generic.go:356] "Generic (PLEG): container finished" podID="9502d995-c7d8-46e9-b1d0-5342b23810d5" containerID="e091e5b16e2aebc5688261a22a354b74beb0254d90df7a269281a5e80de453bf" exitCode=0
Sep 18 15:45:23.526910 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:23.526479    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" event={"ID":"9502d995-c7d8-46e9-b1d0-5342b23810d5","Type":"ContainerDied","Data":"e091e5b16e2aebc5688261a22a354b74beb0254d90df7a269281a5e80de453bf"}
Sep 18 15:45:23.526910 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:23.526826    2585 scope.go:117] "RemoveContainer" containerID="e091e5b16e2aebc5688261a22a354b74beb0254d90df7a269281a5e80de453bf"
Sep 18 15:45:23.527801 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:23.527772    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:45:23.527917 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:23.527874    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:45:23.527981 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:23.527923    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:45:24.503391 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:24.503363    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 15:45:24.537764 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:24.537713    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" event={"ID":"9502d995-c7d8-46e9-b1d0-5342b23810d5","Type":"ContainerStarted","Data":"f4ea1aa9d89fabe9ae7e9c8cc18ae905245bea66243c232441714158e5a1c0be"}
Sep 18 15:45:25.479269 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:25.479213    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:45:25.480455 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:25.480432    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6prf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 15:45:25.480515 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:25.480483    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" podUID="9502d995-c7d8-46e9-b1d0-5342b23810d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 15:45:26.479424 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:26.479387    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6prf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 15:45:26.479806 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:26.479444    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" podUID="9502d995-c7d8-46e9-b1d0-5342b23810d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 15:45:27.479818 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:27.479785    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6prf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 15:45:27.480210 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:27.479863    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" podUID="9502d995-c7d8-46e9-b1d0-5342b23810d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 15:45:28.480429 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:28.480388    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6prf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 15:45:28.480827 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:28.480450    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" podUID="9502d995-c7d8-46e9-b1d0-5342b23810d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 15:45:28.987929 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:28.987893    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:45:29.480202 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:29.480104    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6prf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 15:45:29.480202 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:29.480161    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9" podUID="9502d995-c7d8-46e9-b1d0-5342b23810d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 15:45:30.483358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:30.483327    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:45:30.484394 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:30.484372    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6prf9"
Sep 18 15:45:39.629470 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.629428    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"]
Sep 18 15:45:39.637394 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.637378    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.640965 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.640940    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"]
Sep 18 15:45:39.641328 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.641312    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 15:45:39.641514 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.641497    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 15:45:39.641590 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.641515    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 18 15:45:39.641590 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.641540    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7lm9q\""
Sep 18 15:45:39.762707 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.762680    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.762707 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.762716    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.762979 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.762738    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.762979 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.762753    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.762979 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.762877    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td6dh\" (UniqueName: \"kubernetes.io/projected/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-kube-api-access-td6dh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.762979 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.762916    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.863564 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.863520    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.863755 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.863586    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.863755 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.863614    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.863755 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.863699    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td6dh\" (UniqueName: \"kubernetes.io/projected/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-kube-api-access-td6dh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.863755 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.863733    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.864023 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.863793    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.864023 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.863958    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.864023 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.864008    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.864205 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.864186    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.865957 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.865927    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.866323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.866304    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.871906 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.871884    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td6dh\" (UniqueName: \"kubernetes.io/projected/36504a9e-9c73-4e9f-b7f2-83fc21b50e96-kube-api-access-td6dh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4\" (UID: \"36504a9e-9c73-4e9f-b7f2-83fc21b50e96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:39.948654 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:39.948571    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:40.232199 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:40.232161    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"]
Sep 18 15:45:40.234516 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:45:40.234486    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36504a9e_9c73_4e9f_b7f2_83fc21b50e96.slice/crio-7ec6753b2cdc1f36316624ef7438346e831d44615a80847615a89097846e39ef WatchSource:0}: Error finding container 7ec6753b2cdc1f36316624ef7438346e831d44615a80847615a89097846e39ef: Status 404 returned error can't find the container with id 7ec6753b2cdc1f36316624ef7438346e831d44615a80847615a89097846e39ef
Sep 18 15:45:40.622970 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:40.622930    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4" event={"ID":"36504a9e-9c73-4e9f-b7f2-83fc21b50e96","Type":"ContainerStarted","Data":"7ec6753b2cdc1f36316624ef7438346e831d44615a80847615a89097846e39ef"}
Sep 18 15:45:41.630067 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:41.629985    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4" event={"ID":"36504a9e-9c73-4e9f-b7f2-83fc21b50e96","Type":"ContainerStarted","Data":"bfed76262cb4d4ce15c2344901f4a1574f44cfd0da2eae76c757ab102c30ea6e"}
Sep 18 15:45:41.651636 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:41.651590    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4" podStartSLOduration=1.546950316 podStartE2EDuration="2.651575209s" totalImagesPullingTime="1.104624893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:45:41.649272591 +0000 UTC m=+666.308065253" watchObservedRunningTime="2026-09-18 15:45:41.651575209 +0000 UTC m=+666.310367869"
Sep 18 15:45:49.948942 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:49.948898    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:49.949393 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:49.948959    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:49.961465 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:49.961438    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:45:50.689945 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:45:50.689920    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52wm4"
Sep 18 15:46:23.424128 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.424042    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"]
Sep 18 15:46:23.427384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.427364    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.430052 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.430032    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 15:46:23.439784 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.439762    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"]
Sep 18 15:46:23.539501 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.539471    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.539501 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.539503    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.539741 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.539539    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.539741 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.539620    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.539741 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.539714    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd8hn\" (UniqueName: \"kubernetes.io/projected/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-kube-api-access-gd8hn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.539873 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.539758    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.640725 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.640690    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.640725 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.640724    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.640996 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.640759    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.640996 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.640776    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.640996 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.640833    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd8hn\" (UniqueName: \"kubernetes.io/projected/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-kube-api-access-gd8hn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.640996 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.640904    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.641214 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.641179    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.641214 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.641204    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.641355 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.641327    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.643304 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.643279    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.643465 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.643447    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.649257 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.649235    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd8hn\" (UniqueName: \"kubernetes.io/projected/c603927f-8b31-4aa0-9ec5-c50ab2d32c2d-kube-api-access-gd8hn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6llcc\" (UID: \"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:23.738029 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:23.738001    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:24.009208 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:24.009178    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"]
Sep 18 15:46:24.010559 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:46:24.010529    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc603927f_8b31_4aa0_9ec5_c50ab2d32c2d.slice/crio-b411d66eae691407bbb679a4bddd0045a6695d65116004bed5c121c6e127c402 WatchSource:0}: Error finding container b411d66eae691407bbb679a4bddd0045a6695d65116004bed5c121c6e127c402: Status 404 returned error can't find the container with id b411d66eae691407bbb679a4bddd0045a6695d65116004bed5c121c6e127c402
Sep 18 15:46:24.863309 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:24.863268    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc" event={"ID":"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d","Type":"ContainerStarted","Data":"a454f59fe326e3be31a3c09013095e36ae1de7fd22d952946d2357047246a4be"}
Sep 18 15:46:24.863733 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:24.863330    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc" event={"ID":"c603927f-8b31-4aa0-9ec5-c50ab2d32c2d","Type":"ContainerStarted","Data":"b411d66eae691407bbb679a4bddd0045a6695d65116004bed5c121c6e127c402"}
Sep 18 15:46:24.882072 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:24.882024    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc" podStartSLOduration=1.692527663 podStartE2EDuration="1.882007922s" totalImagesPullingTime="189.480259ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:46:24.879987177 +0000 UTC m=+709.538779863" watchObservedRunningTime="2026-09-18 15:46:24.882007922 +0000 UTC m=+709.540800583"
Sep 18 15:46:33.739110 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:33.739067    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:33.739515 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:33.739140    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:33.751674 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:33.751643    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:46:33.922361 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:46:33.922332    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6llcc"
Sep 18 15:48:08.551103 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.551026    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"]
Sep 18 15:48:08.554421 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.554400    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.557299 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.557276    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-a1f773-openshift-default-dockercfg-wk69h\""
Sep 18 15:48:08.564086 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.564061    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"]
Sep 18 15:48:08.639148 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.639109    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-credential-socket\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.639298 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.639160    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-socket\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.639298 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.639228    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-certs\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.639298 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.639251    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-podinfo\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.639298 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.639274    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-token\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.639434 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.639310    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istiod-ca-cert\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.639434 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.639339    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-data\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.639434 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.639377    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-envoy\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.639527 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.639448    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmqf4\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-kube-api-access-cmqf4\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.705023 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.704991    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"]
Sep 18 15:48:08.709003 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.708978    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.712801 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.711935    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-998130-openshift-default-dockercfg-47wq2\""
Sep 18 15:48:08.725148 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.723074    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"]
Sep 18 15:48:08.740724 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.740694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-credential-socket\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.740886 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.740734    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-socket\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.740886 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.740776    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-certs\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.740886 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.740800    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-podinfo\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.740886 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.740827    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-token\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.740886 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.740875    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istiod-ca-cert\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.741164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.740905    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-data\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.741164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.740947    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-envoy\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.741164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.740985    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmqf4\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-kube-api-access-cmqf4\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.741164 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.741125    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-credential-socket\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.741377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.741229    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-certs\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.741588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.741565    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-socket\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.741818 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.741586    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-data\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.741818 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.741809    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istiod-ca-cert\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.744720 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.744683    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-podinfo\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.745243 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.745226    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-envoy\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.749895 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.749854    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-token\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.749999 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.749946    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmqf4\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-kube-api-access-cmqf4\") pod \"e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") " pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.842509 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.842412    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30764f3a-5b5f-4122-a3be-126ce108115f-istio-podinfo\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.842509 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.842463    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-certs\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.842705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.842533    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-envoy\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.842705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.842616    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-data\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.842705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.842647    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-istio-token\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.842705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.842678    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-socket\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.842876 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.842742    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9cb6\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-kube-api-access-p9cb6\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.842876 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.842821    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30764f3a-5b5f-4122-a3be-126ce108115f-istiod-ca-cert\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.842983 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.842898    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-credential-socket\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.868149 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.868120    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:08.943446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.943410    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30764f3a-5b5f-4122-a3be-126ce108115f-istio-podinfo\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.943580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.943461    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-certs\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.943580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.943488    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-envoy\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.943580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.943532    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-data\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.943580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.943561    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-istio-token\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.943817 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.943590    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-socket\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.943817 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.943629    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9cb6\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-kube-api-access-p9cb6\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.943817 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.943712    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30764f3a-5b5f-4122-a3be-126ce108115f-istiod-ca-cert\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.943817 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.943760    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-credential-socket\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.944611 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.944177    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-credential-socket\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.945534 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.945174    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-certs\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.946017 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.945980    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-socket\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.946268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.946244    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-data\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.947003 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.946973    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30764f3a-5b5f-4122-a3be-126ce108115f-istiod-ca-cert\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.948098 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.948047    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30764f3a-5b5f-4122-a3be-126ce108115f-istio-podinfo\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.950872 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.950679    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-envoy\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.957525 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.957458    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9cb6\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-kube-api-access-p9cb6\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:08.967046 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:08.967024    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-istio-token\") pod \"e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") " pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:09.028967 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.028930    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:09.170073 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.170033    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"]
Sep 18 15:48:09.170788 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:48:09.170761    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefcd7b2_6ea3_4f25_afd1_f183acdd2f89.slice/crio-12e02bdc1b5bb431bb27537d2b290cfd67f6af9f5950d3ae24880a5a2c472b30 WatchSource:0}: Error finding container 12e02bdc1b5bb431bb27537d2b290cfd67f6af9f5950d3ae24880a5a2c472b30: Status 404 returned error can't find the container with id 12e02bdc1b5bb431bb27537d2b290cfd67f6af9f5950d3ae24880a5a2c472b30
Sep 18 15:48:09.173021 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.172990    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:09.173125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.173063    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:09.173125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.173106    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:09.310204 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.310173    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"]
Sep 18 15:48:09.311071 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:48:09.311044    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30764f3a_5b5f_4122_a3be_126ce108115f.slice/crio-6b29888ca87056d5d4ec921283613a977862293474385f24afd80cdf8bc0a6ac WatchSource:0}: Error finding container 6b29888ca87056d5d4ec921283613a977862293474385f24afd80cdf8bc0a6ac: Status 404 returned error can't find the container with id 6b29888ca87056d5d4ec921283613a977862293474385f24afd80cdf8bc0a6ac
Sep 18 15:48:09.313659 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.313626    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:09.313739 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.313704    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:09.313779 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.313747    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:09.431874 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.431735    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" event={"ID":"30764f3a-5b5f-4122-a3be-126ce108115f","Type":"ContainerStarted","Data":"518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5"}
Sep 18 15:48:09.431874 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.431806    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" event={"ID":"30764f3a-5b5f-4122-a3be-126ce108115f","Type":"ContainerStarted","Data":"6b29888ca87056d5d4ec921283613a977862293474385f24afd80cdf8bc0a6ac"}
Sep 18 15:48:09.433659 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.433627    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" event={"ID":"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89","Type":"ContainerStarted","Data":"256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd"}
Sep 18 15:48:09.433804 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.433690    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" event={"ID":"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89","Type":"ContainerStarted","Data":"12e02bdc1b5bb431bb27537d2b290cfd67f6af9f5950d3ae24880a5a2c472b30"}
Sep 18 15:48:09.454703 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.454645    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" podStartSLOduration=1.454623861 podStartE2EDuration="1.454623861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:48:09.451991056 +0000 UTC m=+814.110783717" watchObservedRunningTime="2026-09-18 15:48:09.454623861 +0000 UTC m=+814.113416522"
Sep 18 15:48:09.474223 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.474169    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" podStartSLOduration=1.474145783 podStartE2EDuration="1.474145783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:48:09.472126314 +0000 UTC m=+814.130918975" watchObservedRunningTime="2026-09-18 15:48:09.474145783 +0000 UTC m=+814.132938443"
Sep 18 15:48:09.868627 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:09.868591    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:10.029827 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:10.029797    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:10.034856 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:10.034814    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:10.440210 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:10.440172    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:10.440939 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:10.440921    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:10.873943 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:10.873916    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:11.445811 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:11.445781    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:11.446669 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:11.446648    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:12.691932 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.691900    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"]
Sep 18 15:48:12.695482 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.695461    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.698141 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.698124    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a1f773-serving-cert\""
Sep 18 15:48:12.710768 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.710744    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"]
Sep 18 15:48:12.778854 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.778817    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-metrics-certs\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.779043 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.778951    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a28b90-aa38-40e3-b161-5cc8d23fe824-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.779043 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.779007    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-maas-api-tls\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.779163 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.779061    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6dcq\" (UniqueName: \"kubernetes.io/projected/e7a28b90-aa38-40e3-b161-5cc8d23fe824-kube-api-access-w6dcq\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.880579 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.880542    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a28b90-aa38-40e3-b161-5cc8d23fe824-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.880579 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.880581    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-maas-api-tls\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.880824 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.880613    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6dcq\" (UniqueName: \"kubernetes.io/projected/e7a28b90-aa38-40e3-b161-5cc8d23fe824-kube-api-access-w6dcq\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.880824 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.880676    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-metrics-certs\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.881265 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.881242    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a28b90-aa38-40e3-b161-5cc8d23fe824-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.883078 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.883057    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-metrics-certs\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.883183 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.883105    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-maas-api-tls\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:12.889573 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:12.889550    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6dcq\" (UniqueName: \"kubernetes.io/projected/e7a28b90-aa38-40e3-b161-5cc8d23fe824-kube-api-access-w6dcq\") pod \"maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:13.006008 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.005975    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:13.280611 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.280584    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"]
Sep 18 15:48:13.283296 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:48:13.283265    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a28b90_aa38_40e3_b161_5cc8d23fe824.slice/crio-7223c3c82d40ab766cd473b357ab77f398933f8fafd7b10df2f63ea24427514a WatchSource:0}: Error finding container 7223c3c82d40ab766cd473b357ab77f398933f8fafd7b10df2f63ea24427514a: Status 404 returned error can't find the container with id 7223c3c82d40ab766cd473b357ab77f398933f8fafd7b10df2f63ea24427514a
Sep 18 15:48:13.458345 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.458313    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6" event={"ID":"e7a28b90-aa38-40e3-b161-5cc8d23fe824","Type":"ContainerStarted","Data":"7223c3c82d40ab766cd473b357ab77f398933f8fafd7b10df2f63ea24427514a"}
Sep 18 15:48:13.754619 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.754588    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"]
Sep 18 15:48:13.759498 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.759474    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.762302 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.762274    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-998130-serving-cert\""
Sep 18 15:48:13.775306 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.775277    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"]
Sep 18 15:48:13.891211 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.891170    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dd38-5ae9-459b-9e13-e4b3de039143-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.891363 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.891257    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-metrics-certs\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.891363 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.891303    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff9ks\" (UniqueName: \"kubernetes.io/projected/cbb8dd38-5ae9-459b-9e13-e4b3de039143-kube-api-access-ff9ks\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.891452 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.891431    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-maas-api-tls\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.992117 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.992086    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-maas-api-tls\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.992229 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.992139    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dd38-5ae9-459b-9e13-e4b3de039143-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.992229 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.992224    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-metrics-certs\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.992318 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:48:13.992230    2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-serving-cert: secret "maas-api-e2e-worker-w0-998130-serving-cert" not found
Sep 18 15:48:13.992318 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:48:13.992285    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-maas-api-tls podName:cbb8dd38-5ae9-459b-9e13-e4b3de039143 nodeName:}" failed. No retries permitted until 2026-09-18 15:48:14.492269203 +0000 UTC m=+819.151061862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-maas-api-tls") pod "maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp" (UID: "cbb8dd38-5ae9-459b-9e13-e4b3de039143") : secret "maas-api-e2e-worker-w0-998130-serving-cert" not found
Sep 18 15:48:13.992427 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.992302    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff9ks\" (UniqueName: \"kubernetes.io/projected/cbb8dd38-5ae9-459b-9e13-e4b3de039143-kube-api-access-ff9ks\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.992911 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.992885    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dd38-5ae9-459b-9e13-e4b3de039143-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:13.994742 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:13.994717    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-metrics-certs\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:14.001649 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:14.001612    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff9ks\" (UniqueName: \"kubernetes.io/projected/cbb8dd38-5ae9-459b-9e13-e4b3de039143-kube-api-access-ff9ks\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:14.473081 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:14.473048    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6" event={"ID":"e7a28b90-aa38-40e3-b161-5cc8d23fe824","Type":"ContainerStarted","Data":"f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3"}
Sep 18 15:48:14.473282 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:14.473146    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:14.496491 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:14.496441    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6" podStartSLOduration=1.876574645 podStartE2EDuration="2.496425978s" totalImagesPullingTime="619.851333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:48:14.494099772 +0000 UTC m=+819.152892432" watchObservedRunningTime="2026-09-18 15:48:14.496425978 +0000 UTC m=+819.155218638"
Sep 18 15:48:14.497702 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:14.497667    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-maas-api-tls\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:14.500060 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:14.500038    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-maas-api-tls\") pod \"maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:14.674185 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:14.674148    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:14.960632 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:14.960603    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"]
Sep 18 15:48:14.962382 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:48:14.962351    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb8dd38_5ae9_459b_9e13_e4b3de039143.slice/crio-2848bbe967f4bdecc0903798ecee7fc17f31950e5f2c6d001e308d75cb6c532f WatchSource:0}: Error finding container 2848bbe967f4bdecc0903798ecee7fc17f31950e5f2c6d001e308d75cb6c532f: Status 404 returned error can't find the container with id 2848bbe967f4bdecc0903798ecee7fc17f31950e5f2c6d001e308d75cb6c532f
Sep 18 15:48:15.484023 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:15.483994    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp" event={"ID":"cbb8dd38-5ae9-459b-9e13-e4b3de039143","Type":"ContainerStarted","Data":"2848bbe967f4bdecc0903798ecee7fc17f31950e5f2c6d001e308d75cb6c532f"}
Sep 18 15:48:16.491611 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:16.491573    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp" event={"ID":"cbb8dd38-5ae9-459b-9e13-e4b3de039143","Type":"ContainerStarted","Data":"cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b"}
Sep 18 15:48:16.492081 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:16.491761    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:16.518107 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:16.518056    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp" podStartSLOduration=3.016855715 podStartE2EDuration="3.518041929s" totalImagesPullingTime="501.186214ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:48:16.515172724 +0000 UTC m=+821.173965385" watchObservedRunningTime="2026-09-18 15:48:16.518041929 +0000 UTC m=+821.176834590"
Sep 18 15:48:17.662530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.662499    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"]
Sep 18 15:48:17.666395 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.666372    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.669116 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.669095    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-9792ff-serving-cert\""
Sep 18 15:48:17.680119 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.680094    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"]
Sep 18 15:48:17.727887 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.727858    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-maas-api-tls\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.728037 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.727985    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-metrics-certs\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.728085 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.728029    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btzc4\" (UniqueName: \"kubernetes.io/projected/a71489e6-0651-4a4e-880e-dde1041985f8-kube-api-access-btzc4\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.728124 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.728106    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71489e6-0651-4a4e-880e-dde1041985f8-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.829502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.829464    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-maas-api-tls\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.829660 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.829519    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-metrics-certs\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.829660 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.829538    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btzc4\" (UniqueName: \"kubernetes.io/projected/a71489e6-0651-4a4e-880e-dde1041985f8-kube-api-access-btzc4\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.829660 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.829603    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71489e6-0651-4a4e-880e-dde1041985f8-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.830303 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.830281    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71489e6-0651-4a4e-880e-dde1041985f8-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.832069 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.832045    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-maas-api-tls\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.832153 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.832139    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-metrics-certs\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.853344 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.853323    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btzc4\" (UniqueName: \"kubernetes.io/projected/a71489e6-0651-4a4e-880e-dde1041985f8-kube-api-access-btzc4\") pod \"maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:17.978070 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:17.977993    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:18.263408 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:18.263386    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"]
Sep 18 15:48:18.264929 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:48:18.264905    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71489e6_0651_4a4e_880e_dde1041985f8.slice/crio-b8eb1a9fb3450b0bae5cb387c655f65ba1708eb1268d4476a098df9bc4ad9d43 WatchSource:0}: Error finding container b8eb1a9fb3450b0bae5cb387c655f65ba1708eb1268d4476a098df9bc4ad9d43: Status 404 returned error can't find the container with id b8eb1a9fb3450b0bae5cb387c655f65ba1708eb1268d4476a098df9bc4ad9d43
Sep 18 15:48:18.503894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:18.503859    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2" event={"ID":"a71489e6-0651-4a4e-880e-dde1041985f8","Type":"ContainerStarted","Data":"b8eb1a9fb3450b0bae5cb387c655f65ba1708eb1268d4476a098df9bc4ad9d43"}
Sep 18 15:48:19.542861 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:19.541464    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2" event={"ID":"a71489e6-0651-4a4e-880e-dde1041985f8","Type":"ContainerStarted","Data":"29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463"}
Sep 18 15:48:19.542861 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:19.542410    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:19.578933 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:19.578852    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2" podStartSLOduration=2.237935859 podStartE2EDuration="2.578822595s" totalImagesPullingTime="340.886736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:48:19.573692791 +0000 UTC m=+824.232485451" watchObservedRunningTime="2026-09-18 15:48:19.578822595 +0000 UTC m=+824.237615254"
Sep 18 15:48:20.489869 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:20.489824    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:48:22.502319 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:22.502291    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:48:24.671554 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.671516    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"]
Sep 18 15:48:24.682182 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.682138    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.687022 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.686093    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a8802cb2-serving-cert\""
Sep 18 15:48:24.687022 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.686981    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"]
Sep 18 15:48:24.817145 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.817012    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-openshift-service-ca\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.817145 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.817061    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-maas-api-tls\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.817357 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.817226    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvzx7\" (UniqueName: \"kubernetes.io/projected/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-kube-api-access-jvzx7\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.817357 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.817327    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-metrics-certs\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.918370 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.918332    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-metrics-certs\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.918570 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.918458    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-openshift-service-ca\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.918570 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.918492    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-maas-api-tls\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.918570 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.918561    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvzx7\" (UniqueName: \"kubernetes.io/projected/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-kube-api-access-jvzx7\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.918740 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:48:24.918703    2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-serving-cert: secret "maas-api-e2e-ait-a8802cb2-serving-cert" not found
Sep 18 15:48:24.918794 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:48:24.918782    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-maas-api-tls podName:b3fba09e-6f9d-47a5-aa0c-21f4522c81e0 nodeName:}" failed. No retries permitted until 2026-09-18 15:48:25.418760406 +0000 UTC m=+830.077553048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-maas-api-tls") pod "maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx" (UID: "b3fba09e-6f9d-47a5-aa0c-21f4522c81e0") : secret "maas-api-e2e-ait-a8802cb2-serving-cert" not found
Sep 18 15:48:24.919646 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.919618    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-openshift-service-ca\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.921292 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.921271    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-metrics-certs\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:24.928421 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:24.928358    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvzx7\" (UniqueName: \"kubernetes.io/projected/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-kube-api-access-jvzx7\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:25.423150 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:25.423115    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-maas-api-tls\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:25.425987 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:25.425954    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-maas-api-tls\") pod \"maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:25.601555 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:25.601515    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:25.911777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:25.911360    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"]
Sep 18 15:48:26.483031 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:26.482991    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 15:48:26.483206 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:26.483051    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 15:48:26.588328 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:26.588252    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:48:26.613879 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:26.613820    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx" event={"ID":"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0","Type":"ContainerStarted","Data":"50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04"}
Sep 18 15:48:26.614068 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:26.613907    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx" event={"ID":"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0","Type":"ContainerStarted","Data":"e754501e22cd1229af1f787316ef27f8acc24bcd81548ca32010dfb6c5919c5b"}
Sep 18 15:48:26.614068 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:26.614039    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:26.643718 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:26.643650    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx" podStartSLOduration=2.279316395 podStartE2EDuration="2.643629298s" totalImagesPullingTime="364.312903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:48:26.639035775 +0000 UTC m=+831.297828436" watchObservedRunningTime="2026-09-18 15:48:26.643629298 +0000 UTC m=+831.302421959"
Sep 18 15:48:27.488022 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:27.487978    2585 patch_prober.go:28] interesting pod/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 15:48:27.488462 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:27.488040    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 15:48:28.628115 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.628084    2585 generic.go:356] "Generic (PLEG): container finished" podID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerID="256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd" exitCode=0
Sep 18 15:48:28.628634 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.628161    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" event={"ID":"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89","Type":"ContainerDied","Data":"256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd"}
Sep 18 15:48:28.628634 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.628597    2585 scope.go:117] "RemoveContainer" containerID="256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd"
Sep 18 15:48:28.630006 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.629965    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:28.630125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.630049    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:28.630125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.630100    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:28.630125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.630110    2585 generic.go:356] "Generic (PLEG): container finished" podID="30764f3a-5b5f-4122-a3be-126ce108115f" containerID="518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5" exitCode=0
Sep 18 15:48:28.630288 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.630142    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" event={"ID":"30764f3a-5b5f-4122-a3be-126ce108115f","Type":"ContainerDied","Data":"518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5"}
Sep 18 15:48:28.630485 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.630467    2585 scope.go:117] "RemoveContainer" containerID="518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5"
Sep 18 15:48:28.631406 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.631376    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:28.631506 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.631451    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:28.631506 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:28.631498    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:48:29.642712 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:29.642675    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" event={"ID":"30764f3a-5b5f-4122-a3be-126ce108115f","Type":"ContainerStarted","Data":"f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48"}
Sep 18 15:48:29.651353 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:29.651305    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" event={"ID":"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89","Type":"ContainerStarted","Data":"314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4"}
Sep 18 15:48:29.869375 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:29.869340    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:29.869572 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:29.869457    2585 patch_prober.go:28] interesting pod/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 18 15:48:29.869572 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:29.869493    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 18 15:48:30.029514 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:30.029473    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:30.029725 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:30.029606    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 15:48:30.029725 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:30.029650    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 15:48:30.868542 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:30.868501    2585 patch_prober.go:28] interesting pod/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 18 15:48:30.868974 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:30.868555    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 18 15:48:31.029600 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:31.029559    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 15:48:31.029812 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:31.029631    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 15:48:31.869413 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:31.869376    2585 patch_prober.go:28] interesting pod/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 18 15:48:31.869886 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:31.869434    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 18 15:48:32.029943 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:32.029890    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 15:48:32.030131 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:32.029957    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 15:48:32.635432 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:32.635402    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:32.868588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:32.868548    2585 patch_prober.go:28] interesting pod/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 18 15:48:32.868770 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:32.868617    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 18 15:48:33.029667 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:33.029629    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 15:48:33.030259 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:33.030227    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 15:48:33.869135 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:33.869098    2585 patch_prober.go:28] interesting pod/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 18 15:48:33.869318 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:33.869159    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 18 15:48:34.030413 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:34.030366    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 15:48:34.030941 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:34.030437    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 15:48:34.869518 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:34.869480    2585 patch_prober.go:28] interesting pod/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 18 15:48:34.869697 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:34.869544    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 18 15:48:35.033801 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:35.033769    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:35.034315 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:35.034140    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:35.035031 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:35.035008    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:48:35.872306 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:35.872278    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:35.872585 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:35.872566    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:35.873237 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:35.873217    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:48:46.940780 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:46.940742    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"]
Sep 18 15:48:46.941300 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:46.940965    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx" podUID="b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" containerName="maas-api" containerID="cri-o://50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04" gracePeriod=30
Sep 18 15:48:47.184736 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.184708    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:47.247654 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.247623    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-openshift-service-ca\") pod \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") "
Sep 18 15:48:47.247862 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.247667    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-maas-api-tls\") pod \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") "
Sep 18 15:48:47.247862 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.247755    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-metrics-certs\") pod \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") "
Sep 18 15:48:47.247862 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.247826    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvzx7\" (UniqueName: \"kubernetes.io/projected/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-kube-api-access-jvzx7\") pod \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\" (UID: \"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0\") "
Sep 18 15:48:47.248037 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.247988    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" (UID: "b3fba09e-6f9d-47a5-aa0c-21f4522c81e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:48:47.248200 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.248179    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:48:47.250041 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.250016    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-kube-api-access-jvzx7" (OuterVolumeSpecName: "kube-api-access-jvzx7") pod "b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" (UID: "b3fba09e-6f9d-47a5-aa0c-21f4522c81e0"). InnerVolumeSpecName "kube-api-access-jvzx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:48:47.250297 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.250278    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" (UID: "b3fba09e-6f9d-47a5-aa0c-21f4522c81e0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:48:47.250387 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.250291    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" (UID: "b3fba09e-6f9d-47a5-aa0c-21f4522c81e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:48:47.349152 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.349119    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:48:47.349152 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.349147    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvzx7\" (UniqueName: \"kubernetes.io/projected/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-kube-api-access-jvzx7\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:48:47.349152 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.349158    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:48:47.771883 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.771820    2585 generic.go:356] "Generic (PLEG): container finished" podID="b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" containerID="50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04" exitCode=0
Sep 18 15:48:47.772114 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.772080    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"
Sep 18 15:48:47.772259 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.771970    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx" event={"ID":"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0","Type":"ContainerDied","Data":"50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04"}
Sep 18 15:48:47.772324 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.772268    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx" event={"ID":"b3fba09e-6f9d-47a5-aa0c-21f4522c81e0","Type":"ContainerDied","Data":"e754501e22cd1229af1f787316ef27f8acc24bcd81548ca32010dfb6c5919c5b"}
Sep 18 15:48:47.772324 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.772304    2585 scope.go:117] "RemoveContainer" containerID="50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04"
Sep 18 15:48:47.786242 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.786222    2585 scope.go:117] "RemoveContainer" containerID="50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04"
Sep 18 15:48:47.786741 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:48:47.786713    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04\": container with ID starting with 50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04 not found: ID does not exist" containerID="50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04"
Sep 18 15:48:47.786871 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.786747    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04"} err="failed to get container status \"50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04\": rpc error: code = NotFound desc = could not find container \"50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04\": container with ID starting with 50edac8ff3eb2a7590b175d36d1ce7f39e9ac24fc0f7bdd7b7682d5ae6771c04 not found: ID does not exist"
Sep 18 15:48:47.798499 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.798436    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"]
Sep 18 15:48:47.802395 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:47.802371    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a8802cb2-747c7f6696-t99qx"]
Sep 18 15:48:49.524259 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:49.524225    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" path="/var/lib/kubelet/pods/b3fba09e-6f9d-47a5-aa0c-21f4522c81e0/volumes"
Sep 18 15:48:54.274291 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.274260    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"]
Sep 18 15:48:54.274700 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.274687    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" containerName="maas-api"
Sep 18 15:48:54.274755 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.274703    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" containerName="maas-api"
Sep 18 15:48:54.274798 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.274790    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3fba09e-6f9d-47a5-aa0c-21f4522c81e0" containerName="maas-api"
Sep 18 15:48:54.277921 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.277904    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.281183 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.281160    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-45956207-serving-cert\""
Sep 18 15:48:54.294913 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.294891    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"]
Sep 18 15:48:54.416692 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.416661    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-openshift-service-ca\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.416892 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.416698    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-maas-api-tls\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.416892 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.416785    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6kdn\" (UniqueName: \"kubernetes.io/projected/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-kube-api-access-p6kdn\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.416984 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.416945    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-metrics-certs\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.517992 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.517960    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-metrics-certs\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.518177 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.518018    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-openshift-service-ca\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.518177 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.518039    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-maas-api-tls\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.518177 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:48:54.518135    2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-serving-cert: secret "maas-api-e2e-migrate-45956207-serving-cert" not found
Sep 18 15:48:54.518177 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.518154    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6kdn\" (UniqueName: \"kubernetes.io/projected/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-kube-api-access-p6kdn\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.518414 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:48:54.518203    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-maas-api-tls podName:93139fb1-a9d5-44d8-9e01-6e4ecf38080f nodeName:}" failed. No retries permitted until 2026-09-18 15:48:55.018183621 +0000 UTC m=+859.676976260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-maas-api-tls") pod "maas-api-e2e-migrate-45956207-6dd68b77f9-7d664" (UID: "93139fb1-a9d5-44d8-9e01-6e4ecf38080f") : secret "maas-api-e2e-migrate-45956207-serving-cert" not found
Sep 18 15:48:54.518724 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.518701    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-openshift-service-ca\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.520573 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.520546    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-metrics-certs\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:54.531231 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:54.531177    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6kdn\" (UniqueName: \"kubernetes.io/projected/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-kube-api-access-p6kdn\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:55.023053 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:55.023018    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-maas-api-tls\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:55.025560 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:55.025534    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-maas-api-tls\") pod \"maas-api-e2e-migrate-45956207-6dd68b77f9-7d664\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:55.188797 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:55.188760    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:55.504919 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:55.504896    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"]
Sep 18 15:48:55.506277 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:48:55.506254    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93139fb1_a9d5_44d8_9e01_6e4ecf38080f.slice/crio-741cd18f48963dc0ed7a5501c3b21e8ae1144d8ac1f085046332b5a848d4c61b WatchSource:0}: Error finding container 741cd18f48963dc0ed7a5501c3b21e8ae1144d8ac1f085046332b5a848d4c61b: Status 404 returned error can't find the container with id 741cd18f48963dc0ed7a5501c3b21e8ae1144d8ac1f085046332b5a848d4c61b
Sep 18 15:48:55.820443 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:55.820351    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664" event={"ID":"93139fb1-a9d5-44d8-9e01-6e4ecf38080f","Type":"ContainerStarted","Data":"741cd18f48963dc0ed7a5501c3b21e8ae1144d8ac1f085046332b5a848d4c61b"}
Sep 18 15:48:56.827442 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:56.827398    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664" event={"ID":"93139fb1-a9d5-44d8-9e01-6e4ecf38080f","Type":"ContainerStarted","Data":"eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b"}
Sep 18 15:48:56.827878 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:56.827468    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:48:56.846458 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:48:56.846416    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664" podStartSLOduration=2.503423016 podStartE2EDuration="2.846400959s" totalImagesPullingTime="342.977943ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:48:56.845561231 +0000 UTC m=+861.504353892" watchObservedRunningTime="2026-09-18 15:48:56.846400959 +0000 UTC m=+861.505193619"
Sep 18 15:49:02.838425 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:02.838398    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:49:23.875749 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:23.875667    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"]
Sep 18 15:49:23.876341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:23.875932    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664" podUID="93139fb1-a9d5-44d8-9e01-6e4ecf38080f" containerName="maas-api" containerID="cri-o://eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b" gracePeriod=30
Sep 18 15:49:24.136171 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.136110    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:49:24.201368 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.201334    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-openshift-service-ca\") pod \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") "
Sep 18 15:49:24.201555 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.201495    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-maas-api-tls\") pod \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") "
Sep 18 15:49:24.201555 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.201537    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6kdn\" (UniqueName: \"kubernetes.io/projected/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-kube-api-access-p6kdn\") pod \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") "
Sep 18 15:49:24.201677 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.201575    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-metrics-certs\") pod \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\" (UID: \"93139fb1-a9d5-44d8-9e01-6e4ecf38080f\") "
Sep 18 15:49:24.201730 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.201694    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "93139fb1-a9d5-44d8-9e01-6e4ecf38080f" (UID: "93139fb1-a9d5-44d8-9e01-6e4ecf38080f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:49:24.201973 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.201949    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:49:24.203630 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.203600    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "93139fb1-a9d5-44d8-9e01-6e4ecf38080f" (UID: "93139fb1-a9d5-44d8-9e01-6e4ecf38080f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:49:24.203723 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.203649    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "93139fb1-a9d5-44d8-9e01-6e4ecf38080f" (UID: "93139fb1-a9d5-44d8-9e01-6e4ecf38080f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:49:24.203723 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.203651    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-kube-api-access-p6kdn" (OuterVolumeSpecName: "kube-api-access-p6kdn") pod "93139fb1-a9d5-44d8-9e01-6e4ecf38080f" (UID: "93139fb1-a9d5-44d8-9e01-6e4ecf38080f"). InnerVolumeSpecName "kube-api-access-p6kdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:49:24.303027 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.302992    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:49:24.303027 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.303027    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6kdn\" (UniqueName: \"kubernetes.io/projected/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-kube-api-access-p6kdn\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:49:24.303223 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.303043    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93139fb1-a9d5-44d8-9e01-6e4ecf38080f-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:49:24.991272 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.991234    2585 generic.go:356] "Generic (PLEG): container finished" podID="93139fb1-a9d5-44d8-9e01-6e4ecf38080f" containerID="eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b" exitCode=0
Sep 18 15:49:24.991665 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.991280    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"
Sep 18 15:49:24.991665 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.991321    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664" event={"ID":"93139fb1-a9d5-44d8-9e01-6e4ecf38080f","Type":"ContainerDied","Data":"eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b"}
Sep 18 15:49:24.991665 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.991371    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664" event={"ID":"93139fb1-a9d5-44d8-9e01-6e4ecf38080f","Type":"ContainerDied","Data":"741cd18f48963dc0ed7a5501c3b21e8ae1144d8ac1f085046332b5a848d4c61b"}
Sep 18 15:49:24.991665 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:24.991396    2585 scope.go:117] "RemoveContainer" containerID="eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b"
Sep 18 15:49:25.001820 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:25.001802    2585 scope.go:117] "RemoveContainer" containerID="eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b"
Sep 18 15:49:25.002114 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:49:25.002093    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b\": container with ID starting with eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b not found: ID does not exist" containerID="eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b"
Sep 18 15:49:25.002178 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:25.002127    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b"} err="failed to get container status \"eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b\": rpc error: code = NotFound desc = could not find container \"eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b\": container with ID starting with eda3ffcf06030b0aa3bc420cdd12a2189e2f7893f6a2cde8d05ff0766f95640b not found: ID does not exist"
Sep 18 15:49:25.021986 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:25.021959    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"]
Sep 18 15:49:25.028867 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:25.028830    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45956207-6dd68b77f9-7d664"]
Sep 18 15:49:25.529920 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:25.529885    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93139fb1-a9d5-44d8-9e01-6e4ecf38080f" path="/var/lib/kubelet/pods/93139fb1-a9d5-44d8-9e01-6e4ecf38080f/volumes"
Sep 18 15:49:31.763445 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.763401    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"]
Sep 18 15:49:31.763946 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.763830    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93139fb1-a9d5-44d8-9e01-6e4ecf38080f" containerName="maas-api"
Sep 18 15:49:31.763946 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.763868    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="93139fb1-a9d5-44d8-9e01-6e4ecf38080f" containerName="maas-api"
Sep 18 15:49:31.764098 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.763951    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="93139fb1-a9d5-44d8-9e01-6e4ecf38080f" containerName="maas-api"
Sep 18 15:49:31.768529 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.768507    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.771516 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.771491    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d9680198-serving-cert\""
Sep 18 15:49:31.777814 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.777790    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"]
Sep 18 15:49:31.878261 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.878225    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-metrics-certs\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.878261 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.878269    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3dcf264-c7f1-494f-9352-38caa9003e8e-openshift-service-ca\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.878512 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.878291    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28zwn\" (UniqueName: \"kubernetes.io/projected/f3dcf264-c7f1-494f-9352-38caa9003e8e-kube-api-access-28zwn\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.878512 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.878351    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-maas-api-tls\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.979154 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.979116    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-metrics-certs\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.979410 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.979174    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3dcf264-c7f1-494f-9352-38caa9003e8e-openshift-service-ca\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.979410 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.979200    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28zwn\" (UniqueName: \"kubernetes.io/projected/f3dcf264-c7f1-494f-9352-38caa9003e8e-kube-api-access-28zwn\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.979410 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.979229    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-maas-api-tls\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.979981 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.979950    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3dcf264-c7f1-494f-9352-38caa9003e8e-openshift-service-ca\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.981656 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.981626    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-metrics-certs\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.981762 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.981670    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-maas-api-tls\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:31.987962 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:31.987942    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28zwn\" (UniqueName: \"kubernetes.io/projected/f3dcf264-c7f1-494f-9352-38caa9003e8e-kube-api-access-28zwn\") pod \"maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:32.080793 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:32.080703    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:32.365737 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:32.365706    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"]
Sep 18 15:49:32.366561 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:49:32.366533    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3dcf264_c7f1_494f_9352_38caa9003e8e.slice/crio-dbb00586717ec9e65fb5022501ef4266ef776b20db22f97c0ab645eae0e7973a WatchSource:0}: Error finding container dbb00586717ec9e65fb5022501ef4266ef776b20db22f97c0ab645eae0e7973a: Status 404 returned error can't find the container with id dbb00586717ec9e65fb5022501ef4266ef776b20db22f97c0ab645eae0e7973a
Sep 18 15:49:33.045879 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:33.045819    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6" event={"ID":"f3dcf264-c7f1-494f-9352-38caa9003e8e","Type":"ContainerStarted","Data":"dbb00586717ec9e65fb5022501ef4266ef776b20db22f97c0ab645eae0e7973a"}
Sep 18 15:49:34.052801 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:34.052759    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6" event={"ID":"f3dcf264-c7f1-494f-9352-38caa9003e8e","Type":"ContainerStarted","Data":"7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250"}
Sep 18 15:49:34.053262 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:34.052822    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:49:34.103328 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:34.103275    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6" podStartSLOduration=2.482074622 podStartE2EDuration="3.103260231s" totalImagesPullingTime="621.185609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:49:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:49:34.100179165 +0000 UTC m=+898.758971848" watchObservedRunningTime="2026-09-18 15:49:34.103260231 +0000 UTC m=+898.762052894"
Sep 18 15:49:37.434915 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:37.434884    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:49:37.436151 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:37.436125    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:49:40.064000 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:49:40.063972    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:50:09.809174 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:09.809140    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"]
Sep 18 15:50:09.810158 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:09.810099    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6" podUID="f3dcf264-c7f1-494f-9352-38caa9003e8e" containerName="maas-api" containerID="cri-o://7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250" gracePeriod=30
Sep 18 15:50:10.062204 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.062144    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:50:10.130399 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.130369    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-metrics-certs\") pod \"f3dcf264-c7f1-494f-9352-38caa9003e8e\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") "
Sep 18 15:50:10.130399 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.130403    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-maas-api-tls\") pod \"f3dcf264-c7f1-494f-9352-38caa9003e8e\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") "
Sep 18 15:50:10.130633 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.130487    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28zwn\" (UniqueName: \"kubernetes.io/projected/f3dcf264-c7f1-494f-9352-38caa9003e8e-kube-api-access-28zwn\") pod \"f3dcf264-c7f1-494f-9352-38caa9003e8e\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") "
Sep 18 15:50:10.130633 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.130551    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3dcf264-c7f1-494f-9352-38caa9003e8e-openshift-service-ca\") pod \"f3dcf264-c7f1-494f-9352-38caa9003e8e\" (UID: \"f3dcf264-c7f1-494f-9352-38caa9003e8e\") "
Sep 18 15:50:10.130960 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.130933    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3dcf264-c7f1-494f-9352-38caa9003e8e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3dcf264-c7f1-494f-9352-38caa9003e8e" (UID: "f3dcf264-c7f1-494f-9352-38caa9003e8e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:50:10.132571 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.132519    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f3dcf264-c7f1-494f-9352-38caa9003e8e" (UID: "f3dcf264-c7f1-494f-9352-38caa9003e8e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:50:10.132663 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.132616    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3dcf264-c7f1-494f-9352-38caa9003e8e-kube-api-access-28zwn" (OuterVolumeSpecName: "kube-api-access-28zwn") pod "f3dcf264-c7f1-494f-9352-38caa9003e8e" (UID: "f3dcf264-c7f1-494f-9352-38caa9003e8e"). InnerVolumeSpecName "kube-api-access-28zwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:50:10.132663 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.132626    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3dcf264-c7f1-494f-9352-38caa9003e8e" (UID: "f3dcf264-c7f1-494f-9352-38caa9003e8e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:50:10.232561 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.232459    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:50:10.232561 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.232497    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3dcf264-c7f1-494f-9352-38caa9003e8e-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:50:10.232561 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.232512    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28zwn\" (UniqueName: \"kubernetes.io/projected/f3dcf264-c7f1-494f-9352-38caa9003e8e-kube-api-access-28zwn\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:50:10.232561 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.232534    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3dcf264-c7f1-494f-9352-38caa9003e8e-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:50:10.264562 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.264531    2585 generic.go:356] "Generic (PLEG): container finished" podID="f3dcf264-c7f1-494f-9352-38caa9003e8e" containerID="7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250" exitCode=0
Sep 18 15:50:10.264715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.264581    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"
Sep 18 15:50:10.264715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.264584    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6" event={"ID":"f3dcf264-c7f1-494f-9352-38caa9003e8e","Type":"ContainerDied","Data":"7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250"}
Sep 18 15:50:10.264715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.264692    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6" event={"ID":"f3dcf264-c7f1-494f-9352-38caa9003e8e","Type":"ContainerDied","Data":"dbb00586717ec9e65fb5022501ef4266ef776b20db22f97c0ab645eae0e7973a"}
Sep 18 15:50:10.264715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.264715    2585 scope.go:117] "RemoveContainer" containerID="7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250"
Sep 18 15:50:10.274592 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.274576    2585 scope.go:117] "RemoveContainer" containerID="7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250"
Sep 18 15:50:10.274875 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:50:10.274832    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250\": container with ID starting with 7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250 not found: ID does not exist" containerID="7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250"
Sep 18 15:50:10.274941 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.274879    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250"} err="failed to get container status \"7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250\": rpc error: code = NotFound desc = could not find container \"7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250\": container with ID starting with 7986cbd0542d8063b9b77b94a1083708c118b5f1fe7f3e697dbd29a1c8263250 not found: ID does not exist"
Sep 18 15:50:10.293046 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.293021    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"]
Sep 18 15:50:10.299714 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:10.299690    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9680198-5dc7dfd76c-qv6w6"]
Sep 18 15:50:11.523208 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.523174    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3dcf264-c7f1-494f-9352-38caa9003e8e" path="/var/lib/kubelet/pods/f3dcf264-c7f1-494f-9352-38caa9003e8e/volumes"
Sep 18 15:50:11.871369 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.871291    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"]
Sep 18 15:50:11.871950 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.871933    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3dcf264-c7f1-494f-9352-38caa9003e8e" containerName="maas-api"
Sep 18 15:50:11.872030 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.871952    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3dcf264-c7f1-494f-9352-38caa9003e8e" containerName="maas-api"
Sep 18 15:50:11.872068 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.872037    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3dcf264-c7f1-494f-9352-38caa9003e8e" containerName="maas-api"
Sep 18 15:50:11.876598 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.876578    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:11.879159 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.879140    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-c9ad8682-openshift-default-dockercfg-4klkx\""
Sep 18 15:50:11.895685 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.895661    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"]
Sep 18 15:50:11.949894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.949864    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-data\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:11.950051 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.949910    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkf8b\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-kube-api-access-qkf8b\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:11.950051 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.949947    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-socket\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:11.950051 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.949968    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-podinfo\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:11.950051 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.949996    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-credential-socket\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:11.950051 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.950013    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-envoy\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:11.950051 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.950035    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-token\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:11.950279 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.950129    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istiod-ca-cert\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:11.950279 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:11.950177    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-certs\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.051549 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.051516    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkf8b\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-kube-api-access-qkf8b\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.051745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.051560    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-socket\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.051745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.051582    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-podinfo\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.051745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.051612    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-credential-socket\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.051745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.051629    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-envoy\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.051745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.051645    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-token\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.051745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.051673    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istiod-ca-cert\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.051745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.051722    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-certs\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.052169 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.051759    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-data\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.052169 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.052039    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-socket\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.052169 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.052089    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-credential-socket\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.052329 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.052263    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-certs\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.052463 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.052444    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-data\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.052568 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.052549    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istiod-ca-cert\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.054175 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.054152    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-envoy\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.054521 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.054502    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-podinfo\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.060123 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.060104    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkf8b\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-kube-api-access-qkf8b\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.060422 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.060403    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-token\") pod \"e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") " pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.187227 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.187129    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:12.476091 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.476059    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"]
Sep 18 15:50:12.483664 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.483630    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:50:12.483774 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.483710    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:50:12.483774 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:12.483752    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:50:13.288661 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:13.288624    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" event={"ID":"96214414-4d3b-4d5f-ad3d-3d6e83dafc15","Type":"ContainerStarted","Data":"a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e"}
Sep 18 15:50:13.289100 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:13.288680    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" event={"ID":"96214414-4d3b-4d5f-ad3d-3d6e83dafc15","Type":"ContainerStarted","Data":"15e493f218a06accbf155bb4694f23a6622f2c7381d3af3051c386185c85e8a8"}
Sep 18 15:50:13.317892 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:13.317808    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" podStartSLOduration=2.317788974 podStartE2EDuration="2.317788974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:50:13.313465018 +0000 UTC m=+937.972257656" watchObservedRunningTime="2026-09-18 15:50:13.317788974 +0000 UTC m=+937.976581637"
Sep 18 15:50:14.187747 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:14.187713    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:14.192653 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:14.192627    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:14.294088 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:14.294057    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:14.295070 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:14.295053    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:19.297141 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.297103    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"]
Sep 18 15:50:19.300504 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.300486    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.303349 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.303330    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c9ad8682-serving-cert\""
Sep 18 15:50:19.314332 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.314085    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"]
Sep 18 15:50:19.429091 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.429050    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25lrm\" (UniqueName: \"kubernetes.io/projected/b871ded5-57c8-46dc-89b3-83462ad0746a-kube-api-access-25lrm\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.429296 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.429118    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b871ded5-57c8-46dc-89b3-83462ad0746a-openshift-service-ca\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.429296 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.429156    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-metrics-certs\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.429296 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.429177    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-maas-api-tls\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.530325 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.530288    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25lrm\" (UniqueName: \"kubernetes.io/projected/b871ded5-57c8-46dc-89b3-83462ad0746a-kube-api-access-25lrm\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.530516 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.530340    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b871ded5-57c8-46dc-89b3-83462ad0746a-openshift-service-ca\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.530516 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.530379    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-metrics-certs\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.530516 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.530402    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-maas-api-tls\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.530686 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:50:19.530530    2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-serving-cert: secret "maas-api-e2e-derive-c9ad8682-serving-cert" not found
Sep 18 15:50:19.530686 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:50:19.530594    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-maas-api-tls podName:b871ded5-57c8-46dc-89b3-83462ad0746a nodeName:}" failed. No retries permitted until 2026-09-18 15:50:20.030573359 +0000 UTC m=+944.689366012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-maas-api-tls") pod "maas-api-e2e-derive-c9ad8682-76858d7db-w826x" (UID: "b871ded5-57c8-46dc-89b3-83462ad0746a") : secret "maas-api-e2e-derive-c9ad8682-serving-cert" not found
Sep 18 15:50:19.531147 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.531126    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b871ded5-57c8-46dc-89b3-83462ad0746a-openshift-service-ca\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.532850 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.532824    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-metrics-certs\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:19.540532 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:19.540508    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25lrm\" (UniqueName: \"kubernetes.io/projected/b871ded5-57c8-46dc-89b3-83462ad0746a-kube-api-access-25lrm\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:20.037344 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:20.036591    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-maas-api-tls\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:20.040098 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:20.040069    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-maas-api-tls\") pod \"maas-api-e2e-derive-c9ad8682-76858d7db-w826x\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:20.216021 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:20.215985    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:20.488124 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:20.488095    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"]
Sep 18 15:50:20.489455 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:50:20.489432    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb871ded5_57c8_46dc_89b3_83462ad0746a.slice/crio-8a7e37f8cdeb289c975223a62bfdad072d56f62749e95dcc1f49021b8f34889f WatchSource:0}: Error finding container 8a7e37f8cdeb289c975223a62bfdad072d56f62749e95dcc1f49021b8f34889f: Status 404 returned error can't find the container with id 8a7e37f8cdeb289c975223a62bfdad072d56f62749e95dcc1f49021b8f34889f
Sep 18 15:50:20.490850 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:20.490824    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 15:50:21.337185 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:21.337154    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x" event={"ID":"b871ded5-57c8-46dc-89b3-83462ad0746a","Type":"ContainerStarted","Data":"8a7e37f8cdeb289c975223a62bfdad072d56f62749e95dcc1f49021b8f34889f"}
Sep 18 15:50:22.344362 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:22.344322    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x" event={"ID":"b871ded5-57c8-46dc-89b3-83462ad0746a","Type":"ContainerStarted","Data":"51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f"}
Sep 18 15:50:22.344758 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:22.344431    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:22.368857 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:22.365162    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x" podStartSLOduration=2.545960929 podStartE2EDuration="3.365140648s" totalImagesPullingTime="819.179719ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:50:22.36305105 +0000 UTC m=+947.021843712" watchObservedRunningTime="2026-09-18 15:50:22.365140648 +0000 UTC m=+947.023933309"
Sep 18 15:50:28.355371 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:28.355340    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:40.456830 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:40.456745    2585 generic.go:356] "Generic (PLEG): container finished" podID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerID="a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e" exitCode=0
Sep 18 15:50:40.456830 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:40.456823    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" event={"ID":"96214414-4d3b-4d5f-ad3d-3d6e83dafc15","Type":"ContainerDied","Data":"a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e"}
Sep 18 15:50:40.457273 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:40.457247    2585 scope.go:117] "RemoveContainer" containerID="a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e"
Sep 18 15:50:40.458368 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:40.458338    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:50:40.458476 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:40.458404    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:50:40.458476 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:40.458436    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:50:41.464525 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:41.464488    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" event={"ID":"96214414-4d3b-4d5f-ad3d-3d6e83dafc15","Type":"ContainerStarted","Data":"d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4"}
Sep 18 15:50:42.187706 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:42.187670    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:42.188971 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:42.188946    2585 patch_prober.go:28] interesting pod/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 18 15:50:42.189093 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:42.189039    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 18 15:50:43.187539 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:43.187499    2585 patch_prober.go:28] interesting pod/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 18 15:50:43.187994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:43.187586    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 18 15:50:44.187967 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:44.187932    2585 patch_prober.go:28] interesting pod/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 18 15:50:44.188356 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:44.187990    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 18 15:50:44.294013 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:44.293965    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:45.187900 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:45.187865    2585 patch_prober.go:28] interesting pod/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 18 15:50:45.188066 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:45.187920    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 18 15:50:46.188503 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:46.188468    2585 patch_prober.go:28] interesting pod/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 18 15:50:46.188931 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:46.188527    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 18 15:50:47.191719 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:47.191690    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:47.193016 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:47.192991    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:50:56.764056 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:56.763971    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"]
Sep 18 15:50:56.764513 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:56.764219    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x" podUID="b871ded5-57c8-46dc-89b3-83462ad0746a" containerName="maas-api" containerID="cri-o://51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f" gracePeriod=30
Sep 18 15:50:57.012617 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.012589    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:57.096903 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.096810    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25lrm\" (UniqueName: \"kubernetes.io/projected/b871ded5-57c8-46dc-89b3-83462ad0746a-kube-api-access-25lrm\") pod \"b871ded5-57c8-46dc-89b3-83462ad0746a\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") "
Sep 18 15:50:57.096903 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.096884    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-metrics-certs\") pod \"b871ded5-57c8-46dc-89b3-83462ad0746a\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") "
Sep 18 15:50:57.097131 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.096959    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b871ded5-57c8-46dc-89b3-83462ad0746a-openshift-service-ca\") pod \"b871ded5-57c8-46dc-89b3-83462ad0746a\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") "
Sep 18 15:50:57.097131 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.096993    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-maas-api-tls\") pod \"b871ded5-57c8-46dc-89b3-83462ad0746a\" (UID: \"b871ded5-57c8-46dc-89b3-83462ad0746a\") "
Sep 18 15:50:57.097374 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.097342    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b871ded5-57c8-46dc-89b3-83462ad0746a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b871ded5-57c8-46dc-89b3-83462ad0746a" (UID: "b871ded5-57c8-46dc-89b3-83462ad0746a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:50:57.098982 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.098953    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b871ded5-57c8-46dc-89b3-83462ad0746a-kube-api-access-25lrm" (OuterVolumeSpecName: "kube-api-access-25lrm") pod "b871ded5-57c8-46dc-89b3-83462ad0746a" (UID: "b871ded5-57c8-46dc-89b3-83462ad0746a"). InnerVolumeSpecName "kube-api-access-25lrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:50:57.098982 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.098971    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b871ded5-57c8-46dc-89b3-83462ad0746a" (UID: "b871ded5-57c8-46dc-89b3-83462ad0746a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:50:57.099143 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.099008    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b871ded5-57c8-46dc-89b3-83462ad0746a" (UID: "b871ded5-57c8-46dc-89b3-83462ad0746a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:50:57.198832 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.198789    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:50:57.198832 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.198829    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b871ded5-57c8-46dc-89b3-83462ad0746a-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:50:57.198832 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.198860    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b871ded5-57c8-46dc-89b3-83462ad0746a-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:50:57.199170 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.198872    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25lrm\" (UniqueName: \"kubernetes.io/projected/b871ded5-57c8-46dc-89b3-83462ad0746a-kube-api-access-25lrm\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:50:57.216457 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.215653    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"]
Sep 18 15:50:57.216457 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.215918    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy" containerID="cri-o://d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4" gracePeriod=30
Sep 18 15:50:57.564825 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.564785    2585 generic.go:356] "Generic (PLEG): container finished" podID="b871ded5-57c8-46dc-89b3-83462ad0746a" containerID="51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f" exitCode=0
Sep 18 15:50:57.565024 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.564862    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x" event={"ID":"b871ded5-57c8-46dc-89b3-83462ad0746a","Type":"ContainerDied","Data":"51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f"}
Sep 18 15:50:57.565024 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.564908    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x" event={"ID":"b871ded5-57c8-46dc-89b3-83462ad0746a","Type":"ContainerDied","Data":"8a7e37f8cdeb289c975223a62bfdad072d56f62749e95dcc1f49021b8f34889f"}
Sep 18 15:50:57.565024 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.564936    2585 scope.go:117] "RemoveContainer" containerID="51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f"
Sep 18 15:50:57.565272 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.565104    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"
Sep 18 15:50:57.578454 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.578430    2585 scope.go:117] "RemoveContainer" containerID="51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f"
Sep 18 15:50:57.578813 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:50:57.578778    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f\": container with ID starting with 51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f not found: ID does not exist" containerID="51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f"
Sep 18 15:50:57.578988 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.578818    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f"} err="failed to get container status \"51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f\": rpc error: code = NotFound desc = could not find container \"51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f\": container with ID starting with 51959f53f103b9e5381a33a8e3e2b5b12ac60dbe3ee82789a9567998c410ce5f not found: ID does not exist"
Sep 18 15:50:57.591701 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.591669    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"]
Sep 18 15:50:57.599873 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:57.599847    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c9ad8682-76858d7db-w826x"]
Sep 18 15:50:59.523812 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:50:59.523770    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b871ded5-57c8-46dc-89b3-83462ad0746a" path="/var/lib/kubelet/pods/b871ded5-57c8-46dc-89b3-83462ad0746a/volumes"
Sep 18 15:51:02.194191 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.194150    2585 patch_prober.go:28] interesting pod/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 15:51:02.194567 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.194222    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 15:51:02.568945 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.568917    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:51:02.599220 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.599188    2585 generic.go:356] "Generic (PLEG): container finished" podID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerID="d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4" exitCode=0
Sep 18 15:51:02.599407 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.599236    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" event={"ID":"96214414-4d3b-4d5f-ad3d-3d6e83dafc15","Type":"ContainerDied","Data":"d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4"}
Sep 18 15:51:02.599407 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.599251    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"
Sep 18 15:51:02.599407 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.599286    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps" event={"ID":"96214414-4d3b-4d5f-ad3d-3d6e83dafc15","Type":"ContainerDied","Data":"15e493f218a06accbf155bb4694f23a6622f2c7381d3af3051c386185c85e8a8"}
Sep 18 15:51:02.599407 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.599320    2585 scope.go:117] "RemoveContainer" containerID="d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4"
Sep 18 15:51:02.612576 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.612548    2585 scope.go:117] "RemoveContainer" containerID="a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e"
Sep 18 15:51:02.621983 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.621908    2585 scope.go:117] "RemoveContainer" containerID="d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4"
Sep 18 15:51:02.622262 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:51:02.622232    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4\": container with ID starting with d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4 not found: ID does not exist" containerID="d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4"
Sep 18 15:51:02.622367 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.622270    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4"} err="failed to get container status \"d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4\": rpc error: code = NotFound desc = could not find container \"d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4\": container with ID starting with d2b2c7e736743f413191a34c017c4eaf447577add7c2c62ddb168f82513b69b4 not found: ID does not exist"
Sep 18 15:51:02.622367 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.622288    2585 scope.go:117] "RemoveContainer" containerID="a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e"
Sep 18 15:51:02.622627 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:51:02.622607    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e\": container with ID starting with a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e not found: ID does not exist" containerID="a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e"
Sep 18 15:51:02.622709 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.622631    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e"} err="failed to get container status \"a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e\": rpc error: code = NotFound desc = could not find container \"a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e\": container with ID starting with a4bbe4569bd07584487b566d5f6e4e2ef92d78a92244346c3ee0380104a1ec9e not found: ID does not exist"
Sep 18 15:51:02.656647 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656624    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-credential-socket\") pod \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") "
Sep 18 15:51:02.656800 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656664    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-envoy\") pod \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") "
Sep 18 15:51:02.656800 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656685    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-certs\") pod \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") "
Sep 18 15:51:02.656800 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656711    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkf8b\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-kube-api-access-qkf8b\") pod \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") "
Sep 18 15:51:02.656800 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656770    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-socket\") pod \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") "
Sep 18 15:51:02.657112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656812    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-podinfo\") pod \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") "
Sep 18 15:51:02.657112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656863    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istiod-ca-cert\") pod \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") "
Sep 18 15:51:02.657112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656887    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-token\") pod \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") "
Sep 18 15:51:02.657112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656926    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-data\") pod \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\" (UID: \"96214414-4d3b-4d5f-ad3d-3d6e83dafc15\") "
Sep 18 15:51:02.657112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656950    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96214414-4d3b-4d5f-ad3d-3d6e83dafc15" (UID: "96214414-4d3b-4d5f-ad3d-3d6e83dafc15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:51:02.657112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.656986    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96214414-4d3b-4d5f-ad3d-3d6e83dafc15" (UID: "96214414-4d3b-4d5f-ad3d-3d6e83dafc15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:51:02.657112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.657083    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96214414-4d3b-4d5f-ad3d-3d6e83dafc15" (UID: "96214414-4d3b-4d5f-ad3d-3d6e83dafc15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:51:02.657416 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.657169    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-data" (OuterVolumeSpecName: "istio-data") pod "96214414-4d3b-4d5f-ad3d-3d6e83dafc15" (UID: "96214414-4d3b-4d5f-ad3d-3d6e83dafc15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:51:02.657416 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.657192    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96214414-4d3b-4d5f-ad3d-3d6e83dafc15" (UID: "96214414-4d3b-4d5f-ad3d-3d6e83dafc15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:51:02.657538 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.657424    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:51:02.657538 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.657444    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:51:02.657538 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.657457    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:51:02.657538 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.657471    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:51:02.657538 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.657494    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:51:02.659591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.659529    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96214414-4d3b-4d5f-ad3d-3d6e83dafc15" (UID: "96214414-4d3b-4d5f-ad3d-3d6e83dafc15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 15:51:02.659591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.659547    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-kube-api-access-qkf8b" (OuterVolumeSpecName: "kube-api-access-qkf8b") pod "96214414-4d3b-4d5f-ad3d-3d6e83dafc15" (UID: "96214414-4d3b-4d5f-ad3d-3d6e83dafc15"). InnerVolumeSpecName "kube-api-access-qkf8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:51:02.659737 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.659589    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-token" (OuterVolumeSpecName: "istio-token") pod "96214414-4d3b-4d5f-ad3d-3d6e83dafc15" (UID: "96214414-4d3b-4d5f-ad3d-3d6e83dafc15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:51:02.659737 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.659627    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96214414-4d3b-4d5f-ad3d-3d6e83dafc15" (UID: "96214414-4d3b-4d5f-ad3d-3d6e83dafc15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:51:02.758500 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.758469    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:51:02.758500 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.758499    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkf8b\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-kube-api-access-qkf8b\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:51:02.758694 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.758508    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:51:02.758694 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.758516    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96214414-4d3b-4d5f-ad3d-3d6e83dafc15-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:51:02.924871 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.924832    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"]
Sep 18 15:51:02.931639 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:02.931614    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-c9ad8682-openshift-default-6fd49694dd-jzcps"]
Sep 18 15:51:03.524745 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:03.524713    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" path="/var/lib/kubelet/pods/96214414-4d3b-4d5f-ad3d-3d6e83dafc15/volumes"
Sep 18 15:51:16.743354 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.743315    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"]
Sep 18 15:51:16.743921 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.743904    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy"
Sep 18 15:51:16.743975 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.743925    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy"
Sep 18 15:51:16.743975 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.743942    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy"
Sep 18 15:51:16.743975 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.743951    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy"
Sep 18 15:51:16.744082 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.743978    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b871ded5-57c8-46dc-89b3-83462ad0746a" containerName="maas-api"
Sep 18 15:51:16.744082 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.743987    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b871ded5-57c8-46dc-89b3-83462ad0746a" containerName="maas-api"
Sep 18 15:51:16.744142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.744101    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy"
Sep 18 15:51:16.744142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.744115    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b871ded5-57c8-46dc-89b3-83462ad0746a" containerName="maas-api"
Sep 18 15:51:16.744142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.744125    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="96214414-4d3b-4d5f-ad3d-3d6e83dafc15" containerName="istio-proxy"
Sep 18 15:51:16.747422 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.747402    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.751462 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.751439    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9792ff\"/\"default-dockercfg-6xggs\""
Sep 18 15:51:16.751569 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.751491    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9792ff\"/\"distinct-w3-9792ff-kserve-self-signed-certs\""
Sep 18 15:51:16.751569 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.751517    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9792ff\"/\"openshift-service-ca.crt\""
Sep 18 15:51:16.751569 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.751553    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9792ff\"/\"kube-root-ca.crt\""
Sep 18 15:51:16.763611 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.763589    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"]
Sep 18 15:51:16.794899 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.794872    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-home\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.795043 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.794906    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-tmp-dir\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.795043 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.794938    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-model-cache\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.795043 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.794954    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0672ce54-1368-489c-a172-0160ead37f0f-tls-certs\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.795043 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.795013    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-dshm\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.795043 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.795042    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44q75\" (UniqueName: \"kubernetes.io/projected/0672ce54-1368-489c-a172-0160ead37f0f-kube-api-access-44q75\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.896169 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.896121    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-dshm\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.896479 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.896438    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44q75\" (UniqueName: \"kubernetes.io/projected/0672ce54-1368-489c-a172-0160ead37f0f-kube-api-access-44q75\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.896616 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.896535    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-home\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.896616 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.896566    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-tmp-dir\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.896746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.896635    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-model-cache\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.896746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.896665    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0672ce54-1368-489c-a172-0160ead37f0f-tls-certs\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.897246 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.897218    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-home\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.897334 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.897288    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-model-cache\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.897824 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.897324    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-tmp-dir\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.899010 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.898981    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-dshm\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.905463 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.905441    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0672ce54-1368-489c-a172-0160ead37f0f-tls-certs\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:16.907386 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:16.907361    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44q75\" (UniqueName: \"kubernetes.io/projected/0672ce54-1368-489c-a172-0160ead37f0f-kube-api-access-44q75\") pod \"distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") " pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:17.056738 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:17.056661    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:17.340226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:17.340198    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"]
Sep 18 15:51:17.340852 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:51:17.340821    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0672ce54_1368_489c_a172_0160ead37f0f.slice/crio-9fb414206b76a31b05daf6935ef3c890e5f1a86624a64b063d3b162e6b2fa4ae WatchSource:0}: Error finding container 9fb414206b76a31b05daf6935ef3c890e5f1a86624a64b063d3b162e6b2fa4ae: Status 404 returned error can't find the container with id 9fb414206b76a31b05daf6935ef3c890e5f1a86624a64b063d3b162e6b2fa4ae
Sep 18 15:51:17.687265 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:17.687170    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9" event={"ID":"0672ce54-1368-489c-a172-0160ead37f0f","Type":"ContainerStarted","Data":"45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651"}
Sep 18 15:51:17.687265 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:17.687215    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9" event={"ID":"0672ce54-1368-489c-a172-0160ead37f0f","Type":"ContainerStarted","Data":"9fb414206b76a31b05daf6935ef3c890e5f1a86624a64b063d3b162e6b2fa4ae"}
Sep 18 15:51:17.733066 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:17.733014    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9" podStartSLOduration=1.732998891 podStartE2EDuration="1.732998891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:51:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:51:17.731773561 +0000 UTC m=+1002.390566223" watchObservedRunningTime="2026-09-18 15:51:17.732998891 +0000 UTC m=+1002.391791554"
Sep 18 15:51:27.057337 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:27.057290    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:27.057744 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:27.057365    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:27.070048 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:27.070024    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:51:27.765475 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:51:27.765447    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:52:31.227676 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.227590    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"]
Sep 18 15:52:31.228197 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.227856    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6" podUID="e7a28b90-aa38-40e3-b161-5cc8d23fe824" containerName="maas-api" containerID="cri-o://f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3" gracePeriod=30
Sep 18 15:52:31.705541 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.705515    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:52:31.807228 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.807137    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-metrics-certs\") pod \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") "
Sep 18 15:52:31.807228 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.807182    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6dcq\" (UniqueName: \"kubernetes.io/projected/e7a28b90-aa38-40e3-b161-5cc8d23fe824-kube-api-access-w6dcq\") pod \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") "
Sep 18 15:52:31.807433 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.807258    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-maas-api-tls\") pod \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") "
Sep 18 15:52:31.807433 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.807298    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a28b90-aa38-40e3-b161-5cc8d23fe824-openshift-service-ca\") pod \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\" (UID: \"e7a28b90-aa38-40e3-b161-5cc8d23fe824\") "
Sep 18 15:52:31.807882 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.807829    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a28b90-aa38-40e3-b161-5cc8d23fe824-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e7a28b90-aa38-40e3-b161-5cc8d23fe824" (UID: "e7a28b90-aa38-40e3-b161-5cc8d23fe824"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:52:31.810039 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.810006    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e7a28b90-aa38-40e3-b161-5cc8d23fe824" (UID: "e7a28b90-aa38-40e3-b161-5cc8d23fe824"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:52:31.810142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.810038    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a28b90-aa38-40e3-b161-5cc8d23fe824-kube-api-access-w6dcq" (OuterVolumeSpecName: "kube-api-access-w6dcq") pod "e7a28b90-aa38-40e3-b161-5cc8d23fe824" (UID: "e7a28b90-aa38-40e3-b161-5cc8d23fe824"). InnerVolumeSpecName "kube-api-access-w6dcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:52:31.810142 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.810071    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e7a28b90-aa38-40e3-b161-5cc8d23fe824" (UID: "e7a28b90-aa38-40e3-b161-5cc8d23fe824"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:52:31.908802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.908765    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a28b90-aa38-40e3-b161-5cc8d23fe824-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:31.908802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.908796    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:31.908802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.908806    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6dcq\" (UniqueName: \"kubernetes.io/projected/e7a28b90-aa38-40e3-b161-5cc8d23fe824-kube-api-access-w6dcq\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:31.909052 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:31.908819    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7a28b90-aa38-40e3-b161-5cc8d23fe824-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:32.148512 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:32.148411    2585 generic.go:356] "Generic (PLEG): container finished" podID="e7a28b90-aa38-40e3-b161-5cc8d23fe824" containerID="f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3" exitCode=0
Sep 18 15:52:32.148512 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:32.148463    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"
Sep 18 15:52:32.148512 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:32.148493    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6" event={"ID":"e7a28b90-aa38-40e3-b161-5cc8d23fe824","Type":"ContainerDied","Data":"f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3"}
Sep 18 15:52:32.148810 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:32.148600    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6" event={"ID":"e7a28b90-aa38-40e3-b161-5cc8d23fe824","Type":"ContainerDied","Data":"7223c3c82d40ab766cd473b357ab77f398933f8fafd7b10df2f63ea24427514a"}
Sep 18 15:52:32.148810 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:32.148635    2585 scope.go:117] "RemoveContainer" containerID="f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3"
Sep 18 15:52:32.161473 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:32.161454    2585 scope.go:117] "RemoveContainer" containerID="f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3"
Sep 18 15:52:32.161917 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:52:32.161889    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3\": container with ID starting with f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3 not found: ID does not exist" containerID="f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3"
Sep 18 15:52:32.162005 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:32.161931    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3"} err="failed to get container status \"f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3\": rpc error: code = NotFound desc = could not find container \"f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3\": container with ID starting with f74bf2495aad651c22e1cd02c3de9b1b484179f8c6f9e31af2b19bcb967315d3 not found: ID does not exist"
Sep 18 15:52:32.175483 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:32.175462    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"]
Sep 18 15:52:32.179887 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:32.179855    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a1f773-fdcc565b8-b5rq6"]
Sep 18 15:52:33.523973 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:33.523943    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7a28b90-aa38-40e3-b161-5cc8d23fe824" path="/var/lib/kubelet/pods/e7a28b90-aa38-40e3-b161-5cc8d23fe824/volumes"
Sep 18 15:52:38.735865 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:38.735822    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"]
Sep 18 15:52:38.736333 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:38.736039    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy" containerID="cri-o://314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4" gracePeriod=30
Sep 18 15:52:41.348307 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.348273    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"]
Sep 18 15:52:41.349238 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.348989    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7a28b90-aa38-40e3-b161-5cc8d23fe824" containerName="maas-api"
Sep 18 15:52:41.349238 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.349011    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7a28b90-aa38-40e3-b161-5cc8d23fe824" containerName="maas-api"
Sep 18 15:52:41.349238 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.349119    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7a28b90-aa38-40e3-b161-5cc8d23fe824" containerName="maas-api"
Sep 18 15:52:41.352422 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.352404    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.354964 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.354943    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f84020ef-serving-cert\""
Sep 18 15:52:41.365727 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.365682    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"]
Sep 18 15:52:41.401328 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.401304    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-metrics-certs\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.401441 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.401344    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-maas-api-tls\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.401441 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.401385    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a58f669-8c42-4cad-b1cb-69d97c55dffd-openshift-service-ca\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.401441 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.401431    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6rv7\" (UniqueName: \"kubernetes.io/projected/0a58f669-8c42-4cad-b1cb-69d97c55dffd-kube-api-access-j6rv7\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.501954 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.501920    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-maas-api-tls\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.502109 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.501978    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a58f669-8c42-4cad-b1cb-69d97c55dffd-openshift-service-ca\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.502109 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.502018    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6rv7\" (UniqueName: \"kubernetes.io/projected/0a58f669-8c42-4cad-b1cb-69d97c55dffd-kube-api-access-j6rv7\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.502109 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.502081    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-metrics-certs\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.502109 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:52:41.502081    2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-serving-cert: secret "maas-api-e2e-mt-f84020ef-serving-cert" not found
Sep 18 15:52:41.502286 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:52:41.502140    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-maas-api-tls podName:0a58f669-8c42-4cad-b1cb-69d97c55dffd nodeName:}" failed. No retries permitted until 2026-09-18 15:52:42.002122567 +0000 UTC m=+1086.660915205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-maas-api-tls") pod "maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t" (UID: "0a58f669-8c42-4cad-b1cb-69d97c55dffd") : secret "maas-api-e2e-mt-f84020ef-serving-cert" not found
Sep 18 15:52:41.502737 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.502718    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a58f669-8c42-4cad-b1cb-69d97c55dffd-openshift-service-ca\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.504485 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.504457    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-metrics-certs\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:41.513652 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:41.513627    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6rv7\" (UniqueName: \"kubernetes.io/projected/0a58f669-8c42-4cad-b1cb-69d97c55dffd-kube-api-access-j6rv7\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:42.008146 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:42.008112    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-maas-api-tls\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:42.010738 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:42.010709    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-maas-api-tls\") pod \"maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:42.265198 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:42.265124    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:42.538704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:42.538675    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"]
Sep 18 15:52:42.540041 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:52:42.540017    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a58f669_8c42_4cad_b1cb_69d97c55dffd.slice/crio-19f9cd6bd12d856496fefab4581f187109da249d20e16368c5d1b10f30be23fc WatchSource:0}: Error finding container 19f9cd6bd12d856496fefab4581f187109da249d20e16368c5d1b10f30be23fc: Status 404 returned error can't find the container with id 19f9cd6bd12d856496fefab4581f187109da249d20e16368c5d1b10f30be23fc
Sep 18 15:52:43.214922 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:43.214880    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t" event={"ID":"0a58f669-8c42-4cad-b1cb-69d97c55dffd","Type":"ContainerStarted","Data":"b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0"}
Sep 18 15:52:43.215114 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:43.214948    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t" event={"ID":"0a58f669-8c42-4cad-b1cb-69d97c55dffd","Type":"ContainerStarted","Data":"19f9cd6bd12d856496fefab4581f187109da249d20e16368c5d1b10f30be23fc"}
Sep 18 15:52:43.215114 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:43.215055    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:43.238700 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:43.238647    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t" podStartSLOduration=1.784422466 podStartE2EDuration="2.238598458s" totalImagesPullingTime="454.175992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:52:43.234611599 +0000 UTC m=+1087.893404262" watchObservedRunningTime="2026-09-18 15:52:43.238598458 +0000 UTC m=+1087.897391118"
Sep 18 15:52:43.987712 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:43.987686    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:52:44.029105 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029029    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-token\") pod \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") "
Sep 18 15:52:44.029105 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029096    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-data\") pod \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") "
Sep 18 15:52:44.029314 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029124    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-podinfo\") pod \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") "
Sep 18 15:52:44.029314 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029146    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-certs\") pod \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") "
Sep 18 15:52:44.029314 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029198    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istiod-ca-cert\") pod \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") "
Sep 18 15:52:44.029314 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029234    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-socket\") pod \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") "
Sep 18 15:52:44.029314 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029308    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-credential-socket\") pod \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") "
Sep 18 15:52:44.029595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029341    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-envoy\") pod \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") "
Sep 18 15:52:44.029595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029338    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-data" (OuterVolumeSpecName: "istio-data") pod "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" (UID: "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:52:44.029595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029397    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmqf4\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-kube-api-access-cmqf4\") pod \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\" (UID: \"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89\") "
Sep 18 15:52:44.029595 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029544    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" (UID: "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:52:44.029799 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029599    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" (UID: "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:52:44.029799 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029708    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" (UID: "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:52:44.029799 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029780    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:44.029799 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029796    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:44.030048 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029811    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:44.030048 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.029823    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:44.030048 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.030026    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" (UID: "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:52:44.031535 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.031512    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" (UID: "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 15:52:44.031804 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.031784    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" (UID: "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:52:44.031895 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.031877    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-token" (OuterVolumeSpecName: "istio-token") pod "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" (UID: "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:52:44.031946 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.031904    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-kube-api-access-cmqf4" (OuterVolumeSpecName: "kube-api-access-cmqf4") pod "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" (UID: "fefcd7b2-6ea3-4f25-afd1-f183acdd2f89"). InnerVolumeSpecName "kube-api-access-cmqf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:52:44.130797 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.130762    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:44.130797 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.130792    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:44.131044 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.130804    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:44.131044 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.130816    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:44.131044 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.130827    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmqf4\" (UniqueName: \"kubernetes.io/projected/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89-kube-api-access-cmqf4\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:52:44.223810 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.223776    2585 generic.go:356] "Generic (PLEG): container finished" podID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerID="314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4" exitCode=0
Sep 18 15:52:44.223976 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.223848    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"
Sep 18 15:52:44.223976 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.223865    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" event={"ID":"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89","Type":"ContainerDied","Data":"314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4"}
Sep 18 15:52:44.223976 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.223924    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf" event={"ID":"fefcd7b2-6ea3-4f25-afd1-f183acdd2f89","Type":"ContainerDied","Data":"12e02bdc1b5bb431bb27537d2b290cfd67f6af9f5950d3ae24880a5a2c472b30"}
Sep 18 15:52:44.223976 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.223950    2585 scope.go:117] "RemoveContainer" containerID="314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4"
Sep 18 15:52:44.234222 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.234203    2585 scope.go:117] "RemoveContainer" containerID="256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd"
Sep 18 15:52:44.242367 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.242344    2585 scope.go:117] "RemoveContainer" containerID="314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4"
Sep 18 15:52:44.242636 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:52:44.242616    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4\": container with ID starting with 314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4 not found: ID does not exist" containerID="314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4"
Sep 18 15:52:44.242681 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.242646    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4"} err="failed to get container status \"314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4\": rpc error: code = NotFound desc = could not find container \"314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4\": container with ID starting with 314e79f8f5b8216e0927e2519039af69027b6f7515dd7d8b16e109829990d1b4 not found: ID does not exist"
Sep 18 15:52:44.242681 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.242662    2585 scope.go:117] "RemoveContainer" containerID="256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd"
Sep 18 15:52:44.242905 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:52:44.242889    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd\": container with ID starting with 256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd not found: ID does not exist" containerID="256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd"
Sep 18 15:52:44.242960 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.242910    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd"} err="failed to get container status \"256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd\": rpc error: code = NotFound desc = could not find container \"256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd\": container with ID starting with 256a7c19287f9e62581815527831737efe6306cd9fab5a41180d4b55e1ee30bd not found: ID does not exist"
Sep 18 15:52:44.252806 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.252780    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"]
Sep 18 15:52:44.255382 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:44.255359    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-a1f773-openshift-default-555dd7cf76-9qltf"]
Sep 18 15:52:45.524173 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:45.524143    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" path="/var/lib/kubelet/pods/fefcd7b2-6ea3-4f25-afd1-f183acdd2f89/volumes"
Sep 18 15:52:47.703559 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.703529    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"]
Sep 18 15:52:47.704040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.703939    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy"
Sep 18 15:52:47.704040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.703953    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy"
Sep 18 15:52:47.704040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.703962    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy"
Sep 18 15:52:47.704040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.703968    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy"
Sep 18 15:52:47.704040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.704039    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy"
Sep 18 15:52:47.704208 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.704048    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fefcd7b2-6ea3-4f25-afd1-f183acdd2f89" containerName="istio-proxy"
Sep 18 15:52:47.708544 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.708516    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.711330 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.711284    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-998130\"/\"openshift-service-ca.crt\""
Sep 18 15:52:47.713627 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.713606    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-998130\"/\"kube-root-ca.crt\""
Sep 18 15:52:47.714014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.713830    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-998130\"/\"default-dockercfg-lv7jt\""
Sep 18 15:52:47.714014 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.713904    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-998130\"/\"distinct-2-w0-998130-kserve-self-signed-certs\""
Sep 18 15:52:47.718026 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.718005    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"]
Sep 18 15:52:47.767233 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.767201    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9ss2\" (UniqueName: \"kubernetes.io/projected/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-kube-api-access-r9ss2\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.767415 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.767263    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-home\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.767415 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.767329    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tmp-dir\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.767415 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.767404    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-dshm\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.767558 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.767429    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tls-certs\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.767558 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.767454    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-model-cache\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.867899 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.867864    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tmp-dir\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.868082 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.867924    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-dshm\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.868082 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.868027    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tls-certs\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.868082 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.868070    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-model-cache\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.868264 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.868103    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9ss2\" (UniqueName: \"kubernetes.io/projected/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-kube-api-access-r9ss2\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.868264 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.868151    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-home\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.868377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.868356    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tmp-dir\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.868426 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.868398    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-model-cache\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.868469 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.868437    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-home\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.870134 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.870116    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-dshm\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.870384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.870366    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tls-certs\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:47.876608 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:47.876587    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9ss2\" (UniqueName: \"kubernetes.io/projected/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-kube-api-access-r9ss2\") pod \"distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") " pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:48.021558 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:48.021523    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:48.296564 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:48.296535    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"]
Sep 18 15:52:48.297611 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:52:48.297589    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod828f6c98_b645_4fff_bf28_e4aa47ef5f3e.slice/crio-c5331d8236521fab99649a599912031393a07a8c661c0d7ebc9b28f62e68d97e WatchSource:0}: Error finding container c5331d8236521fab99649a599912031393a07a8c661c0d7ebc9b28f62e68d97e: Status 404 returned error can't find the container with id c5331d8236521fab99649a599912031393a07a8c661c0d7ebc9b28f62e68d97e
Sep 18 15:52:49.228980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:49.228953    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:52:49.260742 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:49.260704    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq" event={"ID":"828f6c98-b645-4fff-bf28-e4aa47ef5f3e","Type":"ContainerStarted","Data":"c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e"}
Sep 18 15:52:49.260920 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:49.260755    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq" event={"ID":"828f6c98-b645-4fff-bf28-e4aa47ef5f3e","Type":"ContainerStarted","Data":"c5331d8236521fab99649a599912031393a07a8c661c0d7ebc9b28f62e68d97e"}
Sep 18 15:52:49.291444 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:49.291395    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq" podStartSLOduration=2.29137927 podStartE2EDuration="2.29137927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:52:49.290159931 +0000 UTC m=+1093.948952591" watchObservedRunningTime="2026-09-18 15:52:49.29137927 +0000 UTC m=+1093.950171929"
Sep 18 15:52:57.914796 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:57.914752    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"]
Sep 18 15:52:57.919282 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:57.919261    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:57.922540 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:57.922521    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-f14aba-openshift-default-dockercfg-2xc9v\""
Sep 18 15:52:57.942360 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:57.942330    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"]
Sep 18 15:52:58.022430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.022390    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:58.022695 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.022672    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:58.043161 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.043136    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:58.066393 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.066364    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-envoy\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.066530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.066401    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-token\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.066530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.066425    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-data\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.066530 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.066513    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxck9\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-kube-api-access-qxck9\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.066641 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.066572    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-socket\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.066641 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.066599    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-credential-socket\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.066641 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.066617    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-certs\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.066775 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.066708    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-podinfo\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.066775 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.066747    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istiod-ca-cert\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168371 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168270    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-token\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168371 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168339    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-data\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168616 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168372    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxck9\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-kube-api-access-qxck9\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168616 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168413    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-socket\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168616 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168438    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-credential-socket\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168616 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168462    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-certs\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168616 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168560    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-podinfo\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168616 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168602    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istiod-ca-cert\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168964 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168668    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-envoy\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168964 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168867    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-data\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168964 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168877    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-socket\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.168964 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.168953    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-credential-socket\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.169151 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.169034    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-certs\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.169377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.169355    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istiod-ca-cert\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.170882 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.170863    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-envoy\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.171210 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.171193    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-podinfo\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.178060 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.178039    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-token\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.178286 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.178270    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxck9\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-kube-api-access-qxck9\") pod \"e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") " pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.235083 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.235048    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:52:58.332196 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.332167    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:52:58.514820 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.514792    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"]
Sep 18 15:52:58.516783 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:52:58.516750    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4dfc6a8_6f30_4936_9278_6190e1de24a5.slice/crio-d473aa7a3dcad6d9a7fd54b2e83b43c72fc7ca4f7e6230d55ab69985101d1f2c WatchSource:0}: Error finding container d473aa7a3dcad6d9a7fd54b2e83b43c72fc7ca4f7e6230d55ab69985101d1f2c: Status 404 returned error can't find the container with id d473aa7a3dcad6d9a7fd54b2e83b43c72fc7ca4f7e6230d55ab69985101d1f2c
Sep 18 15:52:58.518999 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.518969    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:52:58.519101 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.519029    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:52:58.519101 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:58.519055    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:52:59.328339 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:59.328291    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" event={"ID":"b4dfc6a8-6f30-4936-9278-6190e1de24a5","Type":"ContainerStarted","Data":"bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d"}
Sep 18 15:52:59.328798 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:59.328359    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" event={"ID":"b4dfc6a8-6f30-4936-9278-6190e1de24a5","Type":"ContainerStarted","Data":"d473aa7a3dcad6d9a7fd54b2e83b43c72fc7ca4f7e6230d55ab69985101d1f2c"}
Sep 18 15:52:59.353907 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:52:59.353862    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" podStartSLOduration=2.353848662 podStartE2EDuration="2.353848662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:52:59.351786529 +0000 UTC m=+1104.010579190" watchObservedRunningTime="2026-09-18 15:52:59.353848662 +0000 UTC m=+1104.012641312"
Sep 18 15:53:00.235721 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:00.235684    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:53:00.240253 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:00.240228    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:53:00.334450 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:00.334421    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:53:00.335239 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:00.335220    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:53:20.351940 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.346064    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"]
Sep 18 15:53:20.351940 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.349073    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t" podUID="0a58f669-8c42-4cad-b1cb-69d97c55dffd" containerName="maas-api" containerID="cri-o://b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0" gracePeriod=30
Sep 18 15:53:20.643607 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.643325    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:53:20.693523 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.693485    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-metrics-certs\") pod \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") "
Sep 18 15:53:20.693712 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.693539    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-maas-api-tls\") pod \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") "
Sep 18 15:53:20.693712 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.693586    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a58f669-8c42-4cad-b1cb-69d97c55dffd-openshift-service-ca\") pod \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") "
Sep 18 15:53:20.693712 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.693614    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6rv7\" (UniqueName: \"kubernetes.io/projected/0a58f669-8c42-4cad-b1cb-69d97c55dffd-kube-api-access-j6rv7\") pod \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\" (UID: \"0a58f669-8c42-4cad-b1cb-69d97c55dffd\") "
Sep 18 15:53:20.694021 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.693997    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a58f669-8c42-4cad-b1cb-69d97c55dffd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0a58f669-8c42-4cad-b1cb-69d97c55dffd" (UID: "0a58f669-8c42-4cad-b1cb-69d97c55dffd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:53:20.695648 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.695618    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0a58f669-8c42-4cad-b1cb-69d97c55dffd" (UID: "0a58f669-8c42-4cad-b1cb-69d97c55dffd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:53:20.695759 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.695648    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0a58f669-8c42-4cad-b1cb-69d97c55dffd" (UID: "0a58f669-8c42-4cad-b1cb-69d97c55dffd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:53:20.695759 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.695692    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a58f669-8c42-4cad-b1cb-69d97c55dffd-kube-api-access-j6rv7" (OuterVolumeSpecName: "kube-api-access-j6rv7") pod "0a58f669-8c42-4cad-b1cb-69d97c55dffd" (UID: "0a58f669-8c42-4cad-b1cb-69d97c55dffd"). InnerVolumeSpecName "kube-api-access-j6rv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:53:20.795003 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.794968    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:53:20.795003 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.794996    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a58f669-8c42-4cad-b1cb-69d97c55dffd-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:53:20.795003 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.795008    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a58f669-8c42-4cad-b1cb-69d97c55dffd-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:53:20.795237 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:20.795023    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6rv7\" (UniqueName: \"kubernetes.io/projected/0a58f669-8c42-4cad-b1cb-69d97c55dffd-kube-api-access-j6rv7\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:53:21.464219 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.464187    2585 generic.go:356] "Generic (PLEG): container finished" podID="0a58f669-8c42-4cad-b1cb-69d97c55dffd" containerID="b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0" exitCode=0
Sep 18 15:53:21.464725 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.464241    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"
Sep 18 15:53:21.464725 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.464275    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t" event={"ID":"0a58f669-8c42-4cad-b1cb-69d97c55dffd","Type":"ContainerDied","Data":"b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0"}
Sep 18 15:53:21.464725 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.464331    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t" event={"ID":"0a58f669-8c42-4cad-b1cb-69d97c55dffd","Type":"ContainerDied","Data":"19f9cd6bd12d856496fefab4581f187109da249d20e16368c5d1b10f30be23fc"}
Sep 18 15:53:21.464725 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.464353    2585 scope.go:117] "RemoveContainer" containerID="b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0"
Sep 18 15:53:21.474195 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.474138    2585 scope.go:117] "RemoveContainer" containerID="b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0"
Sep 18 15:53:21.474405 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:53:21.474388    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0\": container with ID starting with b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0 not found: ID does not exist" containerID="b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0"
Sep 18 15:53:21.474446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.474413    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0"} err="failed to get container status \"b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0\": rpc error: code = NotFound desc = could not find container \"b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0\": container with ID starting with b9c30e5b9a07605131f6d6e6f7aa74640ba4eecb3a9cf29d3a2915da25691ab0 not found: ID does not exist"
Sep 18 15:53:21.489045 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.489016    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"]
Sep 18 15:53:21.495518 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.495498    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84020ef-6d7db89d46-tvh4t"]
Sep 18 15:53:21.527941 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:21.527911    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a58f669-8c42-4cad-b1cb-69d97c55dffd" path="/var/lib/kubelet/pods/0a58f669-8c42-4cad-b1cb-69d97c55dffd/volumes"
Sep 18 15:53:25.490289 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:25.490256    2585 generic.go:356] "Generic (PLEG): container finished" podID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerID="bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d" exitCode=0
Sep 18 15:53:25.490749 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:25.490330    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" event={"ID":"b4dfc6a8-6f30-4936-9278-6190e1de24a5","Type":"ContainerDied","Data":"bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d"}
Sep 18 15:53:25.490749 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:25.490744    2585 scope.go:117] "RemoveContainer" containerID="bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d"
Sep 18 15:53:25.491829 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:25.491804    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:53:25.491924 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:25.491895    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:53:25.491965 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:25.491930    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:53:26.497727 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:26.497692    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" event={"ID":"b4dfc6a8-6f30-4936-9278-6190e1de24a5","Type":"ContainerStarted","Data":"74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e"}
Sep 18 15:53:27.236207 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:27.236167    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:53:27.237558 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:27.237526    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 18 15:53:27.237705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:27.237589    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 18 15:53:28.235806 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:28.235764    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 18 15:53:28.236268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:28.235861    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 18 15:53:29.236528 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:29.236491    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 18 15:53:29.237157 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:29.237121    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 18 15:53:30.235747 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:30.235709    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 18 15:53:30.236088 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:30.236046    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 18 15:53:30.335437 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:30.335380    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:53:31.236516 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:31.236471    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 18 15:53:31.236710 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:31.236544    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 18 15:53:32.246083 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:32.244650    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:53:32.247368 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:32.247337    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 15:53:32.953098 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:32.953066    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"]
Sep 18 15:53:32.953465 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:32.953453    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a58f669-8c42-4cad-b1cb-69d97c55dffd" containerName="maas-api"
Sep 18 15:53:32.953465 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:32.953467    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a58f669-8c42-4cad-b1cb-69d97c55dffd" containerName="maas-api"
Sep 18 15:53:32.953591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:32.953563    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a58f669-8c42-4cad-b1cb-69d97c55dffd" containerName="maas-api"
Sep 18 15:53:32.956496 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:32.956476    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:32.960737 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:32.960715    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-4e81aa-serving-cert\""
Sep 18 15:53:32.971275 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:32.971252    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"]
Sep 18 15:53:33.021193 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.021155    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.021384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.021218    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.021384 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.021322    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9kb4\" (UniqueName: \"kubernetes.io/projected/da985123-a3b8-4781-96b2-43484984ee22-kube-api-access-s9kb4\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.021485 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.021403    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da985123-a3b8-4781-96b2-43484984ee22-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.122036 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.121997    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9kb4\" (UniqueName: \"kubernetes.io/projected/da985123-a3b8-4781-96b2-43484984ee22-kube-api-access-s9kb4\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.122233 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.122064    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da985123-a3b8-4781-96b2-43484984ee22-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.122233 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.122128    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.122233 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.122207    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.122363 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:53:33.122340    2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-serving-cert: secret "maas-api-e2e-shared-b-w2-4e81aa-serving-cert" not found
Sep 18 15:53:33.122462 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:53:33.122447    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-maas-api-tls podName:da985123-a3b8-4781-96b2-43484984ee22 nodeName:}" failed. No retries permitted until 2026-09-18 15:53:33.622394883 +0000 UTC m=+1138.281187522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-maas-api-tls") pod "maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7" (UID: "da985123-a3b8-4781-96b2-43484984ee22") : secret "maas-api-e2e-shared-b-w2-4e81aa-serving-cert" not found
Sep 18 15:53:33.122712 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.122694    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da985123-a3b8-4781-96b2-43484984ee22-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.124776 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.124754    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.134103 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.134074    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9kb4\" (UniqueName: \"kubernetes.io/projected/da985123-a3b8-4781-96b2-43484984ee22-kube-api-access-s9kb4\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.627485 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.627448    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.630075 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.630055    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:33.868415 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:33.868380    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:34.150468 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:34.150441    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"]
Sep 18 15:53:34.151979 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:53:34.151957    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda985123_a3b8_4781_96b2_43484984ee22.slice/crio-cae52a5cd91a74623639c185923bf2f745f346a94a8b260e1ffa7235334d7499 WatchSource:0}: Error finding container cae52a5cd91a74623639c185923bf2f745f346a94a8b260e1ffa7235334d7499: Status 404 returned error can't find the container with id cae52a5cd91a74623639c185923bf2f745f346a94a8b260e1ffa7235334d7499
Sep 18 15:53:34.562412 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:34.562387    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7" event={"ID":"da985123-a3b8-4781-96b2-43484984ee22","Type":"ContainerStarted","Data":"cae52a5cd91a74623639c185923bf2f745f346a94a8b260e1ffa7235334d7499"}
Sep 18 15:53:35.569552 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:35.569514    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7" event={"ID":"da985123-a3b8-4781-96b2-43484984ee22","Type":"ContainerStarted","Data":"3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e"}
Sep 18 15:53:35.569954 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:35.569664    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:35.591192 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:35.591143    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7" podStartSLOduration=3.232118188 podStartE2EDuration="3.59112812s" totalImagesPullingTime="359.009932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:53:35.589336961 +0000 UTC m=+1140.248129622" watchObservedRunningTime="2026-09-18 15:53:35.59112812 +0000 UTC m=+1140.249920780"
Sep 18 15:53:41.581070 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:41.581040    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 15:53:43.770055 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.770020    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"]
Sep 18 15:53:43.777184 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.777158    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:43.780247 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.780225    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-195881-openshift-default-dockercfg-v5qvs\""
Sep 18 15:53:43.787894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.787864    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"]
Sep 18 15:53:43.929264 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.929228    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpncc\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-kube-api-access-qpncc\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:43.929264 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.929267    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-credential-socket\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:43.929527 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.929358    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-token\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:43.929527 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.929390    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istiod-ca-cert\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:43.929527 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.929410    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-certs\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:43.929527 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.929488    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-envoy\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:43.929748 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.929529    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-data\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:43.929748 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.929555    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-podinfo\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:43.929748 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:43.929577    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-socket\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.030765 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.030673    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-token\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.030765 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.030717    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istiod-ca-cert\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.030765 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.030741    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-certs\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031076 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.030876    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-envoy\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031076 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.030916    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-data\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031076 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.031052    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-podinfo\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.031092    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-socket\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.031126    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-certs\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.031194    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpncc\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-kube-api-access-qpncc\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031268 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.031228    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-credential-socket\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031483 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.031228    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-data\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031483 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.031439    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-socket\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031592 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.031540    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istiod-ca-cert\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.031653 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.031599    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-credential-socket\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.033222 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.033196    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-envoy\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.033432 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.033414    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-podinfo\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.045938 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.045917    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-token\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.046084 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.046026    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpncc\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-kube-api-access-qpncc\") pod \"e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") " pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.089178 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.089147    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:44.368695 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.368668    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"]
Sep 18 15:53:44.370724 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:53:44.370694    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d01582_9f91_4fca_81e6_c0fe9a3dde29.slice/crio-bff23c24a9ed467c64cb0024870315116b8e429400db8293f937a94f2b86664e WatchSource:0}: Error finding container bff23c24a9ed467c64cb0024870315116b8e429400db8293f937a94f2b86664e: Status 404 returned error can't find the container with id bff23c24a9ed467c64cb0024870315116b8e429400db8293f937a94f2b86664e
Sep 18 15:53:44.373016 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.372985    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:53:44.373088 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.373051    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:53:44.373088 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.373078    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:53:44.622597 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.622504    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" event={"ID":"24d01582-9f91-4fca-81e6-c0fe9a3dde29","Type":"ContainerStarted","Data":"325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73"}
Sep 18 15:53:44.622597 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.622569    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" event={"ID":"24d01582-9f91-4fca-81e6-c0fe9a3dde29","Type":"ContainerStarted","Data":"bff23c24a9ed467c64cb0024870315116b8e429400db8293f937a94f2b86664e"}
Sep 18 15:53:44.649504 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:44.649416    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" podStartSLOduration=1.649394677 podStartE2EDuration="1.649394677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:53:44.645425462 +0000 UTC m=+1149.304218146" watchObservedRunningTime="2026-09-18 15:53:44.649394677 +0000 UTC m=+1149.308187337"
Sep 18 15:53:45.089772 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:45.089737    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:45.095124 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:45.095098    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:45.631451 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:45.631423    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:53:45.632448 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:53:45.632430    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:54:16.673994 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:16.673953    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 15:54:16.674408 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:16.674016    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 15:54:18.828296 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:18.828210    2585 generic.go:356] "Generic (PLEG): container finished" podID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerID="325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73" exitCode=0
Sep 18 15:54:18.828296 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:18.828279    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" event={"ID":"24d01582-9f91-4fca-81e6-c0fe9a3dde29","Type":"ContainerDied","Data":"325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73"}
Sep 18 15:54:18.828763 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:18.828748    2585 scope.go:117] "RemoveContainer" containerID="325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73"
Sep 18 15:54:18.829894 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:18.829860    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:54:18.829967 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:18.829931    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:54:18.829967 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:18.829963    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:54:19.839144 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:19.839104    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" event={"ID":"24d01582-9f91-4fca-81e6-c0fe9a3dde29","Type":"ContainerStarted","Data":"b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc"}
Sep 18 15:54:20.090191 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:20.090105    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:54:20.090427 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:20.090402    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 15:54:20.090544 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:20.090518    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 15:54:21.090062 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:21.090027    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 15:54:21.090475 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:21.090081    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 15:54:22.090112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:22.090078    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 15:54:22.090517 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:22.090134    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 15:54:23.090112 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:23.090078    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 15:54:23.090317 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:23.090148    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 15:54:24.089561 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.089530    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 15:54:24.089741 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.089596    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 15:54:24.758952 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.758916    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"]
Sep 18 15:54:24.763052 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.762731    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.765936 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.765914    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-655d9c-serving-cert\""
Sep 18 15:54:24.775117 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.775095    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"]
Sep 18 15:54:24.862927 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.862879    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.863106 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.862954    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.863106 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.863028    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qj45\" (UniqueName: \"kubernetes.io/projected/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-kube-api-access-4qj45\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.863106 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.863095    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.964134 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.964090    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.964342 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.964151    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.964342 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.964178    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qj45\" (UniqueName: \"kubernetes.io/projected/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-kube-api-access-4qj45\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.964342 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.964220    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.964828 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.964803    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.966777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.966747    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.966977 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.966952    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:24.975245 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:24.975210    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qj45\" (UniqueName: \"kubernetes.io/projected/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-kube-api-access-4qj45\") pod \"maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:25.074907 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:25.074808    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:25.148257 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:25.146615    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:54:25.148257 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:25.147226    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:54:25.150019 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:25.149965    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 15:54:25.364663 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:25.364621    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"]
Sep 18 15:54:25.366335 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:54:25.366302    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf0ed09_ead9_43b2_a4d6_4103ca2f6bc1.slice/crio-d399c33c3a36716d5141ea62c57785711abdddaefe25a61402e85f83c59efbca WatchSource:0}: Error finding container d399c33c3a36716d5141ea62c57785711abdddaefe25a61402e85f83c59efbca: Status 404 returned error can't find the container with id d399c33c3a36716d5141ea62c57785711abdddaefe25a61402e85f83c59efbca
Sep 18 15:54:25.879633 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:25.879593    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm" event={"ID":"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1","Type":"ContainerStarted","Data":"47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99"}
Sep 18 15:54:25.879978 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:25.879695    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm" event={"ID":"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1","Type":"ContainerStarted","Data":"d399c33c3a36716d5141ea62c57785711abdddaefe25a61402e85f83c59efbca"}
Sep 18 15:54:25.879978 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:25.879867    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:25.899430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:25.899379    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm" podStartSLOduration=1.447549742 podStartE2EDuration="1.899364214s" totalImagesPullingTime="451.814472ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:54:25.897123293 +0000 UTC m=+1190.555915966" watchObservedRunningTime="2026-09-18 15:54:25.899364214 +0000 UTC m=+1190.558156900"
Sep 18 15:54:31.890891 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:31.890830    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 15:54:37.474596 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:37.474566    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:54:37.477571 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:54:37.477552    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:55:00.042100 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.042048    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"]
Sep 18 15:55:00.046073 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.046053    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.048783 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.048755    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c56bfef6-openshift-default-dockercfg-ctxgz\""
Sep 18 15:55:00.057770 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.057741    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"]
Sep 18 15:55:00.112303 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.112267    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-podinfo\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.112502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.112338    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-data\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.112502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.112361    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-envoy\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.112502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.112393    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-socket\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.112502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.112417    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-587fg\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-kube-api-access-587fg\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.112502 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.112455    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istiod-ca-cert\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.112715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.112531    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-certs\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.112715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.112585    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-credential-socket\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.112715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.112610    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-token\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213257 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213221    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-credential-socket\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213452 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213264    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-token\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213452 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213307    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-podinfo\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213452 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213366    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-data\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213452 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213424    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-envoy\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213700 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213496    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-socket\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213700 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213528    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-587fg\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-kube-api-access-587fg\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213700 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213578    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istiod-ca-cert\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213700 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213655    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-certs\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213940 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213729    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-credential-socket\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.213940 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213758    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-data\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.214070 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.213974    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-certs\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.214129 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.214081    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-socket\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.214399 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.214374    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istiod-ca-cert\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.216368 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.216336    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-envoy\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.216672 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.216650    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-podinfo\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.225125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.225101    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-token\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.225439 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.225419    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-587fg\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-kube-api-access-587fg\") pod \"e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") " pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.358316 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.358219    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:00.467059 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.467024    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"]
Sep 18 15:55:00.662820 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.662798    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"]
Sep 18 15:55:00.665425 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:55:00.665396    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e62f7b7_36bb_48e2_98b3_98ba655f51d8.slice/crio-fb2cf7761a48b5d1f5b76af9d0f15d24db5c0ccbe0c21ffc97146bbe32554ab9 WatchSource:0}: Error finding container fb2cf7761a48b5d1f5b76af9d0f15d24db5c0ccbe0c21ffc97146bbe32554ab9: Status 404 returned error can't find the container with id fb2cf7761a48b5d1f5b76af9d0f15d24db5c0ccbe0c21ffc97146bbe32554ab9
Sep 18 15:55:00.667953 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.667920    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:55:00.668074 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.667988    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:55:00.668074 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:00.668013    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:55:01.107461 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:01.107428    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" event={"ID":"3e62f7b7-36bb-48e2-98b3-98ba655f51d8","Type":"ContainerStarted","Data":"31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e"}
Sep 18 15:55:01.107877 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:01.107480    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" event={"ID":"3e62f7b7-36bb-48e2-98b3-98ba655f51d8","Type":"ContainerStarted","Data":"fb2cf7761a48b5d1f5b76af9d0f15d24db5c0ccbe0c21ffc97146bbe32554ab9"}
Sep 18 15:55:01.129847 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:01.129791    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podStartSLOduration=1.129777352 podStartE2EDuration="1.129777352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:55:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:55:01.126737916 +0000 UTC m=+1225.785530577" watchObservedRunningTime="2026-09-18 15:55:01.129777352 +0000 UTC m=+1225.788570012"
Sep 18 15:55:01.359389 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:01.359308    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:01.364322 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:01.364300    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:02.113270 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:02.113235    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:02.114269 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:02.114251    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:09.256183 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.256148    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"]
Sep 18 15:55:09.261288 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.260764    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.263364 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.263339    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c56bfef6-serving-cert\""
Sep 18 15:55:09.271491 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.270488    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"]
Sep 18 15:55:09.303643 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.303617    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-metrics-certs\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.303781 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.303675    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-maas-api-tls\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.303781 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.303761    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29s6r\" (UniqueName: \"kubernetes.io/projected/9ddd6583-997b-4500-a2fe-5fe32e9362c9-kube-api-access-29s6r\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.303937 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.303821    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd6583-997b-4500-a2fe-5fe32e9362c9-openshift-service-ca\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.405125 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.405082    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-metrics-certs\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.405308 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.405153    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-maas-api-tls\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.405308 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.405199    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29s6r\" (UniqueName: \"kubernetes.io/projected/9ddd6583-997b-4500-a2fe-5fe32e9362c9-kube-api-access-29s6r\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.405308 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.405246    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd6583-997b-4500-a2fe-5fe32e9362c9-openshift-service-ca\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.405308 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:55:09.405296    2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-serving-cert: secret "maas-api-e2e-mt-c56bfef6-serving-cert" not found
Sep 18 15:55:09.405479 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:55:09.405374    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-maas-api-tls podName:9ddd6583-997b-4500-a2fe-5fe32e9362c9 nodeName:}" failed. No retries permitted until 2026-09-18 15:55:09.905356926 +0000 UTC m=+1234.564149563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-maas-api-tls") pod "maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq" (UID: "9ddd6583-997b-4500-a2fe-5fe32e9362c9") : secret "maas-api-e2e-mt-c56bfef6-serving-cert" not found
Sep 18 15:55:09.405965 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.405939    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd6583-997b-4500-a2fe-5fe32e9362c9-openshift-service-ca\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.407686 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.407664    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-metrics-certs\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.421218 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.421193    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29s6r\" (UniqueName: \"kubernetes.io/projected/9ddd6583-997b-4500-a2fe-5fe32e9362c9-kube-api-access-29s6r\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.911909 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.911877    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-maas-api-tls\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:09.914413 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:09.914384    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-maas-api-tls\") pod \"maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:10.176618 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:10.176531    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:10.450591 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:10.450559    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"]
Sep 18 15:55:10.451626 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:55:10.451597    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ddd6583_997b_4500_a2fe_5fe32e9362c9.slice/crio-830f34162cd045a949a40d655416d9cff447d014bf768880f1f3e4389d7aff7b WatchSource:0}: Error finding container 830f34162cd045a949a40d655416d9cff447d014bf768880f1f3e4389d7aff7b: Status 404 returned error can't find the container with id 830f34162cd045a949a40d655416d9cff447d014bf768880f1f3e4389d7aff7b
Sep 18 15:55:11.177272 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:11.177184    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq" event={"ID":"9ddd6583-997b-4500-a2fe-5fe32e9362c9","Type":"ContainerStarted","Data":"9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81"}
Sep 18 15:55:11.177272 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:11.177242    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq" event={"ID":"9ddd6583-997b-4500-a2fe-5fe32e9362c9","Type":"ContainerStarted","Data":"830f34162cd045a949a40d655416d9cff447d014bf768880f1f3e4389d7aff7b"}
Sep 18 15:55:11.177498 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:11.177299    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:11.201815 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:11.201762    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq" podStartSLOduration=1.835027498 podStartE2EDuration="2.20174757s" totalImagesPullingTime="366.720072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:55:11.200254062 +0000 UTC m=+1235.859046723" watchObservedRunningTime="2026-09-18 15:55:11.20174757 +0000 UTC m=+1235.860540229"
Sep 18 15:55:14.137832 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.137798    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"]
Sep 18 15:55:14.138358 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.138021    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2" podUID="a71489e6-0651-4a4e-880e-dde1041985f8" containerName="maas-api" containerID="cri-o://29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463" gracePeriod=30
Sep 18 15:55:14.398020 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.397954    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:55:14.464185 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.464151    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-metrics-certs\") pod \"a71489e6-0651-4a4e-880e-dde1041985f8\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") "
Sep 18 15:55:14.464185 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.464188    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71489e6-0651-4a4e-880e-dde1041985f8-openshift-service-ca\") pod \"a71489e6-0651-4a4e-880e-dde1041985f8\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") "
Sep 18 15:55:14.464417 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.464214    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btzc4\" (UniqueName: \"kubernetes.io/projected/a71489e6-0651-4a4e-880e-dde1041985f8-kube-api-access-btzc4\") pod \"a71489e6-0651-4a4e-880e-dde1041985f8\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") "
Sep 18 15:55:14.464417 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.464361    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-maas-api-tls\") pod \"a71489e6-0651-4a4e-880e-dde1041985f8\" (UID: \"a71489e6-0651-4a4e-880e-dde1041985f8\") "
Sep 18 15:55:14.464658 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.464628    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a71489e6-0651-4a4e-880e-dde1041985f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a71489e6-0651-4a4e-880e-dde1041985f8" (UID: "a71489e6-0651-4a4e-880e-dde1041985f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:55:14.464815 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.464743    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71489e6-0651-4a4e-880e-dde1041985f8-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:14.466439 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.466411    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a71489e6-0651-4a4e-880e-dde1041985f8-kube-api-access-btzc4" (OuterVolumeSpecName: "kube-api-access-btzc4") pod "a71489e6-0651-4a4e-880e-dde1041985f8" (UID: "a71489e6-0651-4a4e-880e-dde1041985f8"). InnerVolumeSpecName "kube-api-access-btzc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:55:14.466588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.466470    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a71489e6-0651-4a4e-880e-dde1041985f8" (UID: "a71489e6-0651-4a4e-880e-dde1041985f8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:55:14.466588 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.466492    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a71489e6-0651-4a4e-880e-dde1041985f8" (UID: "a71489e6-0651-4a4e-880e-dde1041985f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:55:14.565578 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.565538    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:14.565810 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.565793    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btzc4\" (UniqueName: \"kubernetes.io/projected/a71489e6-0651-4a4e-880e-dde1041985f8-kube-api-access-btzc4\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:14.565932 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:14.565916    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71489e6-0651-4a4e-880e-dde1041985f8-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:15.202526 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.202494    2585 generic.go:356] "Generic (PLEG): container finished" podID="a71489e6-0651-4a4e-880e-dde1041985f8" containerID="29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463" exitCode=0
Sep 18 15:55:15.202957 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.202543    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2" event={"ID":"a71489e6-0651-4a4e-880e-dde1041985f8","Type":"ContainerDied","Data":"29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463"}
Sep 18 15:55:15.202957 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.202547    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"
Sep 18 15:55:15.202957 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.202583    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2" event={"ID":"a71489e6-0651-4a4e-880e-dde1041985f8","Type":"ContainerDied","Data":"b8eb1a9fb3450b0bae5cb387c655f65ba1708eb1268d4476a098df9bc4ad9d43"}
Sep 18 15:55:15.202957 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.202604    2585 scope.go:117] "RemoveContainer" containerID="29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463"
Sep 18 15:55:15.213202 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.213154    2585 scope.go:117] "RemoveContainer" containerID="29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463"
Sep 18 15:55:15.213523 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:55:15.213479    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463\": container with ID starting with 29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463 not found: ID does not exist" containerID="29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463"
Sep 18 15:55:15.213622 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.213545    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463"} err="failed to get container status \"29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463\": rpc error: code = NotFound desc = could not find container \"29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463\": container with ID starting with 29d3b6d1b53d4f09309646e7509b667107c0b6cc3d7103530b5d68523a056463 not found: ID does not exist"
Sep 18 15:55:15.227612 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.227587    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"]
Sep 18 15:55:15.234419 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.234397    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-9792ff-7987bd5846-t5jc2"]
Sep 18 15:55:15.487089 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.487052    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9" podUID="0672ce54-1368-489c-a172-0160ead37f0f" containerName="main" containerID="cri-o://45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651" gracePeriod=45
Sep 18 15:55:15.523148 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.523119    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a71489e6-0651-4a4e-880e-dde1041985f8" path="/var/lib/kubelet/pods/a71489e6-0651-4a4e-880e-dde1041985f8/volumes"
Sep 18 15:55:15.748777 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.748724    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:55:15.879123 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879093    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-tmp-dir\") pod \"0672ce54-1368-489c-a172-0160ead37f0f\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") "
Sep 18 15:55:15.879282 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879174    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-dshm\") pod \"0672ce54-1368-489c-a172-0160ead37f0f\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") "
Sep 18 15:55:15.879282 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879196    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0672ce54-1368-489c-a172-0160ead37f0f-tls-certs\") pod \"0672ce54-1368-489c-a172-0160ead37f0f\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") "
Sep 18 15:55:15.879371 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879294    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-home\") pod \"0672ce54-1368-489c-a172-0160ead37f0f\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") "
Sep 18 15:55:15.879371 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879361    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-model-cache\") pod \"0672ce54-1368-489c-a172-0160ead37f0f\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") "
Sep 18 15:55:15.879456 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879367    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0672ce54-1368-489c-a172-0160ead37f0f" (UID: "0672ce54-1368-489c-a172-0160ead37f0f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:55:15.879456 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879396    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44q75\" (UniqueName: \"kubernetes.io/projected/0672ce54-1368-489c-a172-0160ead37f0f-kube-api-access-44q75\") pod \"0672ce54-1368-489c-a172-0160ead37f0f\" (UID: \"0672ce54-1368-489c-a172-0160ead37f0f\") "
Sep 18 15:55:15.879627 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879583    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0672ce54-1368-489c-a172-0160ead37f0f" (UID: "0672ce54-1368-489c-a172-0160ead37f0f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:55:15.879794 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879775    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-model-cache\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:15.879900 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879802    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-tmp-dir\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:15.879900 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.879858    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-home" (OuterVolumeSpecName: "home") pod "0672ce54-1368-489c-a172-0160ead37f0f" (UID: "0672ce54-1368-489c-a172-0160ead37f0f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:55:15.881380 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.881358    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-dshm" (OuterVolumeSpecName: "dshm") pod "0672ce54-1368-489c-a172-0160ead37f0f" (UID: "0672ce54-1368-489c-a172-0160ead37f0f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:55:15.881508 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.881456    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0672ce54-1368-489c-a172-0160ead37f0f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0672ce54-1368-489c-a172-0160ead37f0f" (UID: "0672ce54-1368-489c-a172-0160ead37f0f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:55:15.881572 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.881511    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0672ce54-1368-489c-a172-0160ead37f0f-kube-api-access-44q75" (OuterVolumeSpecName: "kube-api-access-44q75") pod "0672ce54-1368-489c-a172-0160ead37f0f" (UID: "0672ce54-1368-489c-a172-0160ead37f0f"). InnerVolumeSpecName "kube-api-access-44q75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:55:15.980698 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.980669    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44q75\" (UniqueName: \"kubernetes.io/projected/0672ce54-1368-489c-a172-0160ead37f0f-kube-api-access-44q75\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:15.980698 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.980695    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-dshm\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:15.980698 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.980705    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0672ce54-1368-489c-a172-0160ead37f0f-tls-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:15.980940 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:15.980714    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0672ce54-1368-489c-a172-0160ead37f0f-home\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:55:16.211414 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:16.211322    2585 generic.go:356] "Generic (PLEG): container finished" podID="0672ce54-1368-489c-a172-0160ead37f0f" containerID="45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651" exitCode=0
Sep 18 15:55:16.211414 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:16.211369    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"
Sep 18 15:55:16.211414 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:16.211402    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9" event={"ID":"0672ce54-1368-489c-a172-0160ead37f0f","Type":"ContainerDied","Data":"45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651"}
Sep 18 15:55:16.212040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:16.211443    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9" event={"ID":"0672ce54-1368-489c-a172-0160ead37f0f","Type":"ContainerDied","Data":"9fb414206b76a31b05daf6935ef3c890e5f1a86624a64b063d3b162e6b2fa4ae"}
Sep 18 15:55:16.212040 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:16.211459    2585 scope.go:117] "RemoveContainer" containerID="45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651"
Sep 18 15:55:16.221714 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:16.221691    2585 scope.go:117] "RemoveContainer" containerID="45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651"
Sep 18 15:55:16.222006 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:55:16.221989    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651\": container with ID starting with 45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651 not found: ID does not exist" containerID="45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651"
Sep 18 15:55:16.222058 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:16.222014    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651"} err="failed to get container status \"45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651\": rpc error: code = NotFound desc = could not find container \"45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651\": container with ID starting with 45dcf96883beafbf8e1158af4f2b595580348937aaadc56a968b70014127f651 not found: ID does not exist"
Sep 18 15:55:16.243056 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:16.243033    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"]
Sep 18 15:55:16.248017 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:16.247993    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9792ff/distinct-w3-9792ff-kserve-5699d8cf4c-m4xl9"]
Sep 18 15:55:17.187819 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:17.187789    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:55:17.524497 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:17.524461    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0672ce54-1368-489c-a172-0160ead37f0f" path="/var/lib/kubelet/pods/0672ce54-1368-489c-a172-0160ead37f0f/volumes"
Sep 18 15:55:33.156004 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:33.155961    2585 patch_prober.go:28] interesting pod/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 15:55:33.156381 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:33.156024    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 15:55:34.323080 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:34.323049    2585 generic.go:356] "Generic (PLEG): container finished" podID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerID="31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e" exitCode=0
Sep 18 15:55:34.323080 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:34.323086    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" event={"ID":"3e62f7b7-36bb-48e2-98b3-98ba655f51d8","Type":"ContainerDied","Data":"31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e"}
Sep 18 15:55:34.323554 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:34.323497    2585 scope.go:117] "RemoveContainer" containerID="31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e"
Sep 18 15:55:34.324696 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:34.324671    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:55:34.324750 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:34.324738    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:55:34.324786 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:34.324769    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:55:35.336035 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:35.335986    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" event={"ID":"3e62f7b7-36bb-48e2-98b3-98ba655f51d8","Type":"ContainerStarted","Data":"a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278"}
Sep 18 15:55:35.359344 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:35.359316    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:35.359529 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:35.359509    2585 patch_prober.go:28] interesting pod/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 18 15:55:35.359602 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:35.359563    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 18 15:55:36.359472 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:36.359440    2585 patch_prober.go:28] interesting pod/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 18 15:55:36.359900 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:36.359492    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 18 15:55:37.359330 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:37.359297    2585 patch_prober.go:28] interesting pod/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 18 15:55:37.359507 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:37.359375    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 18 15:55:38.359051 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:38.359014    2585 patch_prober.go:28] interesting pod/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 18 15:55:38.359228 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:38.359072    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 18 15:55:39.358609 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:39.358574    2585 patch_prober.go:28] interesting pod/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 18 15:55:39.359032 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:39.358631    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 18 15:55:40.359438 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:40.359396    2585 patch_prober.go:28] interesting pod/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 18 15:55:40.360054 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:40.359475    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 18 15:55:41.362469 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:41.362431    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:41.362873 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:41.362711    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:55:41.363426 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:55:41.363409    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:57:14.818230 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:14.818193    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"]
Sep 18 15:57:14.818724 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:14.818404    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq" podUID="9ddd6583-997b-4500-a2fe-5fe32e9362c9" containerName="maas-api" containerID="cri-o://9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81" gracePeriod=30
Sep 18 15:57:15.064887 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.064861    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:57:15.175169 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.175062    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-maas-api-tls\") pod \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") "
Sep 18 15:57:15.175169 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.175159    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd6583-997b-4500-a2fe-5fe32e9362c9-openshift-service-ca\") pod \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") "
Sep 18 15:57:15.175412 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.175260    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29s6r\" (UniqueName: \"kubernetes.io/projected/9ddd6583-997b-4500-a2fe-5fe32e9362c9-kube-api-access-29s6r\") pod \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") "
Sep 18 15:57:15.175412 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.175295    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-metrics-certs\") pod \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\" (UID: \"9ddd6583-997b-4500-a2fe-5fe32e9362c9\") "
Sep 18 15:57:15.175536 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.175509    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ddd6583-997b-4500-a2fe-5fe32e9362c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ddd6583-997b-4500-a2fe-5fe32e9362c9" (UID: "9ddd6583-997b-4500-a2fe-5fe32e9362c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:57:15.175685 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.175663    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddd6583-997b-4500-a2fe-5fe32e9362c9-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:15.177430 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.177402    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ddd6583-997b-4500-a2fe-5fe32e9362c9" (UID: "9ddd6583-997b-4500-a2fe-5fe32e9362c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:57:15.177517 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.177425    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9ddd6583-997b-4500-a2fe-5fe32e9362c9" (UID: "9ddd6583-997b-4500-a2fe-5fe32e9362c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:57:15.177560 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.177520    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ddd6583-997b-4500-a2fe-5fe32e9362c9-kube-api-access-29s6r" (OuterVolumeSpecName: "kube-api-access-29s6r") pod "9ddd6583-997b-4500-a2fe-5fe32e9362c9" (UID: "9ddd6583-997b-4500-a2fe-5fe32e9362c9"). InnerVolumeSpecName "kube-api-access-29s6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:57:15.276895 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.276822    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:15.276895 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.276898    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29s6r\" (UniqueName: \"kubernetes.io/projected/9ddd6583-997b-4500-a2fe-5fe32e9362c9-kube-api-access-29s6r\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:15.277124 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.276912    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddd6583-997b-4500-a2fe-5fe32e9362c9-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:15.937226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.937128    2585 generic.go:356] "Generic (PLEG): container finished" podID="9ddd6583-997b-4500-a2fe-5fe32e9362c9" containerID="9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81" exitCode=0
Sep 18 15:57:15.937226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.937190    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"
Sep 18 15:57:15.937226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.937214    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq" event={"ID":"9ddd6583-997b-4500-a2fe-5fe32e9362c9","Type":"ContainerDied","Data":"9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81"}
Sep 18 15:57:15.937806 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.937280    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq" event={"ID":"9ddd6583-997b-4500-a2fe-5fe32e9362c9","Type":"ContainerDied","Data":"830f34162cd045a949a40d655416d9cff447d014bf768880f1f3e4389d7aff7b"}
Sep 18 15:57:15.937806 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.937312    2585 scope.go:117] "RemoveContainer" containerID="9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81"
Sep 18 15:57:15.946990 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.946974    2585 scope.go:117] "RemoveContainer" containerID="9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81"
Sep 18 15:57:15.947271 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:57:15.947251    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81\": container with ID starting with 9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81 not found: ID does not exist" containerID="9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81"
Sep 18 15:57:15.947325 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.947278    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81"} err="failed to get container status \"9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81\": rpc error: code = NotFound desc = could not find container \"9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81\": container with ID starting with 9910267866e1f1f0c9729132c5e8aa21ed3a187806c5b4bbfaf5a01868760c81 not found: ID does not exist"
Sep 18 15:57:15.955024 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.954999    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"]
Sep 18 15:57:15.958261 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:15.958236    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c56bfef6-5d5d564c48-frckq"]
Sep 18 15:57:17.524348 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:17.524315    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ddd6583-997b-4500-a2fe-5fe32e9362c9" path="/var/lib/kubelet/pods/9ddd6583-997b-4500-a2fe-5fe32e9362c9/volumes"
Sep 18 15:57:39.109257 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:39.109223    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"]
Sep 18 15:57:50.005925 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.005890    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"]
Sep 18 15:57:50.006561 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.006105    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp" podUID="cbb8dd38-5ae9-459b-9e13-e4b3de039143" containerName="maas-api" containerID="cri-o://cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b" gracePeriod=30
Sep 18 15:57:50.479419 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.479393    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:57:50.638683 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.638589    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff9ks\" (UniqueName: \"kubernetes.io/projected/cbb8dd38-5ae9-459b-9e13-e4b3de039143-kube-api-access-ff9ks\") pod \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") "
Sep 18 15:57:50.638683 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.638680    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-metrics-certs\") pod \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") "
Sep 18 15:57:50.638980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.638713    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-maas-api-tls\") pod \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") "
Sep 18 15:57:50.638980 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.638752    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dd38-5ae9-459b-9e13-e4b3de039143-openshift-service-ca\") pod \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\" (UID: \"cbb8dd38-5ae9-459b-9e13-e4b3de039143\") "
Sep 18 15:57:50.639176 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.639142    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb8dd38-5ae9-459b-9e13-e4b3de039143-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cbb8dd38-5ae9-459b-9e13-e4b3de039143" (UID: "cbb8dd38-5ae9-459b-9e13-e4b3de039143"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:57:50.640755 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.640723    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cbb8dd38-5ae9-459b-9e13-e4b3de039143" (UID: "cbb8dd38-5ae9-459b-9e13-e4b3de039143"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:57:50.640893 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.640765    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb8dd38-5ae9-459b-9e13-e4b3de039143-kube-api-access-ff9ks" (OuterVolumeSpecName: "kube-api-access-ff9ks") pod "cbb8dd38-5ae9-459b-9e13-e4b3de039143" (UID: "cbb8dd38-5ae9-459b-9e13-e4b3de039143"). InnerVolumeSpecName "kube-api-access-ff9ks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:57:50.640893 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.640761    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cbb8dd38-5ae9-459b-9e13-e4b3de039143" (UID: "cbb8dd38-5ae9-459b-9e13-e4b3de039143"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:57:50.739656 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.739607    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:50.739656 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.739638    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dd38-5ae9-459b-9e13-e4b3de039143-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:50.739656 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.739654    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dd38-5ae9-459b-9e13-e4b3de039143-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:50.739939 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:50.739667    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff9ks\" (UniqueName: \"kubernetes.io/projected/cbb8dd38-5ae9-459b-9e13-e4b3de039143-kube-api-access-ff9ks\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:51.146318 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.146283    2585 generic.go:356] "Generic (PLEG): container finished" podID="cbb8dd38-5ae9-459b-9e13-e4b3de039143" containerID="cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b" exitCode=0
Sep 18 15:57:51.146746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.146337    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"
Sep 18 15:57:51.146746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.146378    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp" event={"ID":"cbb8dd38-5ae9-459b-9e13-e4b3de039143","Type":"ContainerDied","Data":"cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b"}
Sep 18 15:57:51.146746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.146462    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp" event={"ID":"cbb8dd38-5ae9-459b-9e13-e4b3de039143","Type":"ContainerDied","Data":"2848bbe967f4bdecc0903798ecee7fc17f31950e5f2c6d001e308d75cb6c532f"}
Sep 18 15:57:51.146746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.146485    2585 scope.go:117] "RemoveContainer" containerID="cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b"
Sep 18 15:57:51.159720 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.159698    2585 scope.go:117] "RemoveContainer" containerID="cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b"
Sep 18 15:57:51.160020 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:57:51.159991    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b\": container with ID starting with cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b not found: ID does not exist" containerID="cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b"
Sep 18 15:57:51.160118 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.160024    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b"} err="failed to get container status \"cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b\": rpc error: code = NotFound desc = could not find container \"cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b\": container with ID starting with cd26e3ed5daf1119c3406dd91a77a38f35c21a49d5862085a34e2efee6af197b not found: ID does not exist"
Sep 18 15:57:51.187746 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.187715    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"]
Sep 18 15:57:51.195303 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.195275    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-998130-744cb9dcb8-v7czp"]
Sep 18 15:57:51.525302 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:51.525270    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbb8dd38-5ae9-459b-9e13-e4b3de039143" path="/var/lib/kubelet/pods/cbb8dd38-5ae9-459b-9e13-e4b3de039143/volumes"
Sep 18 15:57:54.129124 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.129081    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq" podUID="828f6c98-b645-4fff-bf28-e4aa47ef5f3e" containerName="main" containerID="cri-o://c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e" gracePeriod=45
Sep 18 15:57:54.399651 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.399627    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:57:54.476765 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.476738    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9ss2\" (UniqueName: \"kubernetes.io/projected/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-kube-api-access-r9ss2\") pod \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") "
Sep 18 15:57:54.476967 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.476872    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-home\") pod \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") "
Sep 18 15:57:54.476967 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.476930    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-model-cache\") pod \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") "
Sep 18 15:57:54.476967 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.476961    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-dshm\") pod \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") "
Sep 18 15:57:54.477136 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.476994    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tls-certs\") pod \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") "
Sep 18 15:57:54.477136 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.477015    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tmp-dir\") pod \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\" (UID: \"828f6c98-b645-4fff-bf28-e4aa47ef5f3e\") "
Sep 18 15:57:54.477249 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.477140    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-home" (OuterVolumeSpecName: "home") pod "828f6c98-b645-4fff-bf28-e4aa47ef5f3e" (UID: "828f6c98-b645-4fff-bf28-e4aa47ef5f3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:57:54.477249 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.477229    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-model-cache" (OuterVolumeSpecName: "model-cache") pod "828f6c98-b645-4fff-bf28-e4aa47ef5f3e" (UID: "828f6c98-b645-4fff-bf28-e4aa47ef5f3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:57:54.477375 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.477348    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "828f6c98-b645-4fff-bf28-e4aa47ef5f3e" (UID: "828f6c98-b645-4fff-bf28-e4aa47ef5f3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:57:54.477438 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.477404    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-home\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:54.477438 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.477424    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-model-cache\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:54.479444 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.479404    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-kube-api-access-r9ss2" (OuterVolumeSpecName: "kube-api-access-r9ss2") pod "828f6c98-b645-4fff-bf28-e4aa47ef5f3e" (UID: "828f6c98-b645-4fff-bf28-e4aa47ef5f3e"). InnerVolumeSpecName "kube-api-access-r9ss2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:57:54.479569 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.479520    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-dshm" (OuterVolumeSpecName: "dshm") pod "828f6c98-b645-4fff-bf28-e4aa47ef5f3e" (UID: "828f6c98-b645-4fff-bf28-e4aa47ef5f3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:57:54.479775 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.479756    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "828f6c98-b645-4fff-bf28-e4aa47ef5f3e" (UID: "828f6c98-b645-4fff-bf28-e4aa47ef5f3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 15:57:54.578995 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.578965    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-dshm\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:54.578995 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.579010    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tls-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:54.579196 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.579040    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-tmp-dir\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:54.579196 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:54.579048    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9ss2\" (UniqueName: \"kubernetes.io/projected/828f6c98-b645-4fff-bf28-e4aa47ef5f3e-kube-api-access-r9ss2\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:57:55.171880 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.171833    2585 generic.go:356] "Generic (PLEG): container finished" podID="828f6c98-b645-4fff-bf28-e4aa47ef5f3e" containerID="c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e" exitCode=0
Sep 18 15:57:55.172398 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.171890    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"
Sep 18 15:57:55.172398 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.171893    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq" event={"ID":"828f6c98-b645-4fff-bf28-e4aa47ef5f3e","Type":"ContainerDied","Data":"c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e"}
Sep 18 15:57:55.172398 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.171997    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq" event={"ID":"828f6c98-b645-4fff-bf28-e4aa47ef5f3e","Type":"ContainerDied","Data":"c5331d8236521fab99649a599912031393a07a8c661c0d7ebc9b28f62e68d97e"}
Sep 18 15:57:55.172398 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.172025    2585 scope.go:117] "RemoveContainer" containerID="c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e"
Sep 18 15:57:55.184192 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.184145    2585 scope.go:117] "RemoveContainer" containerID="c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e"
Sep 18 15:57:55.184685 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:57:55.184639    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e\": container with ID starting with c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e not found: ID does not exist" containerID="c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e"
Sep 18 15:57:55.184788 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.184695    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e"} err="failed to get container status \"c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e\": rpc error: code = NotFound desc = could not find container \"c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e\": container with ID starting with c5c77eacecde97b3cb1ad153c5d1d931ecd534466d925a06f27fa93b94d48e9e not found: ID does not exist"
Sep 18 15:57:55.199474 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.199452    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"]
Sep 18 15:57:55.202869 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.202849    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-998130/distinct-2-w0-998130-kserve-6cc4bf448b-xjzzq"]
Sep 18 15:57:55.523775 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:55.523744    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="828f6c98-b645-4fff-bf28-e4aa47ef5f3e" path="/var/lib/kubelet/pods/828f6c98-b645-4fff-bf28-e4aa47ef5f3e/volumes"
Sep 18 15:57:57.218561 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:57.218521    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"]
Sep 18 15:57:57.218997 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:57:57.218738    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy" containerID="cri-o://f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48" gracePeriod=30
Sep 18 15:58:02.471637 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.471612    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:58:02.659127 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659096    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9cb6\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-kube-api-access-p9cb6\") pod \"30764f3a-5b5f-4122-a3be-126ce108115f\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") "
Sep 18 15:58:02.659276 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659148    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30764f3a-5b5f-4122-a3be-126ce108115f-istiod-ca-cert\") pod \"30764f3a-5b5f-4122-a3be-126ce108115f\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") "
Sep 18 15:58:02.659276 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659177    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-certs\") pod \"30764f3a-5b5f-4122-a3be-126ce108115f\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") "
Sep 18 15:58:02.659276 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659229    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-credential-socket\") pod \"30764f3a-5b5f-4122-a3be-126ce108115f\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") "
Sep 18 15:58:02.659276 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659258    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-envoy\") pod \"30764f3a-5b5f-4122-a3be-126ce108115f\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") "
Sep 18 15:58:02.659501 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659288    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-istio-token\") pod \"30764f3a-5b5f-4122-a3be-126ce108115f\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") "
Sep 18 15:58:02.659501 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659349    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30764f3a-5b5f-4122-a3be-126ce108115f-istio-podinfo\") pod \"30764f3a-5b5f-4122-a3be-126ce108115f\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") "
Sep 18 15:58:02.659501 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659436    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-data\") pod \"30764f3a-5b5f-4122-a3be-126ce108115f\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") "
Sep 18 15:58:02.659501 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659468    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-socket\") pod \"30764f3a-5b5f-4122-a3be-126ce108115f\" (UID: \"30764f3a-5b5f-4122-a3be-126ce108115f\") "
Sep 18 15:58:02.659710 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659587    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30764f3a-5b5f-4122-a3be-126ce108115f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30764f3a-5b5f-4122-a3be-126ce108115f" (UID: "30764f3a-5b5f-4122-a3be-126ce108115f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:58:02.659710 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659651    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30764f3a-5b5f-4122-a3be-126ce108115f" (UID: "30764f3a-5b5f-4122-a3be-126ce108115f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:02.659826 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659731    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-data" (OuterVolumeSpecName: "istio-data") pod "30764f3a-5b5f-4122-a3be-126ce108115f" (UID: "30764f3a-5b5f-4122-a3be-126ce108115f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:02.659913 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659863    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30764f3a-5b5f-4122-a3be-126ce108115f" (UID: "30764f3a-5b5f-4122-a3be-126ce108115f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:02.659974 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659907    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:02.659974 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659930    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30764f3a-5b5f-4122-a3be-126ce108115f-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:02.659974 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.659943    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:02.660375 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.660335    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30764f3a-5b5f-4122-a3be-126ce108115f" (UID: "30764f3a-5b5f-4122-a3be-126ce108115f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:02.661487 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.661453    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-kube-api-access-p9cb6" (OuterVolumeSpecName: "kube-api-access-p9cb6") pod "30764f3a-5b5f-4122-a3be-126ce108115f" (UID: "30764f3a-5b5f-4122-a3be-126ce108115f"). InnerVolumeSpecName "kube-api-access-p9cb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:58:02.661885 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.661863    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-istio-token" (OuterVolumeSpecName: "istio-token") pod "30764f3a-5b5f-4122-a3be-126ce108115f" (UID: "30764f3a-5b5f-4122-a3be-126ce108115f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:58:02.662081 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.662067    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30764f3a-5b5f-4122-a3be-126ce108115f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30764f3a-5b5f-4122-a3be-126ce108115f" (UID: "30764f3a-5b5f-4122-a3be-126ce108115f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 15:58:02.662159 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.662145    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30764f3a-5b5f-4122-a3be-126ce108115f" (UID: "30764f3a-5b5f-4122-a3be-126ce108115f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:02.761330 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.761303    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:02.761330 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.761327    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:02.761484 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.761336    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30764f3a-5b5f-4122-a3be-126ce108115f-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:02.761484 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.761346    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:02.761484 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.761356    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9cb6\" (UniqueName: \"kubernetes.io/projected/30764f3a-5b5f-4122-a3be-126ce108115f-kube-api-access-p9cb6\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:02.761484 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:02.761366    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30764f3a-5b5f-4122-a3be-126ce108115f-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:03.224896 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.224861    2585 generic.go:356] "Generic (PLEG): container finished" podID="30764f3a-5b5f-4122-a3be-126ce108115f" containerID="f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48" exitCode=0
Sep 18 15:58:03.225102 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.224916    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"
Sep 18 15:58:03.225102 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.224934    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" event={"ID":"30764f3a-5b5f-4122-a3be-126ce108115f","Type":"ContainerDied","Data":"f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48"}
Sep 18 15:58:03.225102 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.224987    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t" event={"ID":"30764f3a-5b5f-4122-a3be-126ce108115f","Type":"ContainerDied","Data":"6b29888ca87056d5d4ec921283613a977862293474385f24afd80cdf8bc0a6ac"}
Sep 18 15:58:03.225102 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.225012    2585 scope.go:117] "RemoveContainer" containerID="f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48"
Sep 18 15:58:03.235166 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.235152    2585 scope.go:117] "RemoveContainer" containerID="518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5"
Sep 18 15:58:03.243740 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.243727    2585 scope.go:117] "RemoveContainer" containerID="f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48"
Sep 18 15:58:03.244061 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:58:03.244031    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48\": container with ID starting with f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48 not found: ID does not exist" containerID="f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48"
Sep 18 15:58:03.244154 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.244073    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48"} err="failed to get container status \"f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48\": rpc error: code = NotFound desc = could not find container \"f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48\": container with ID starting with f65c7c84d44c2a8a70ddc21159c3207c682ad82c2a2d4194f697565c66614d48 not found: ID does not exist"
Sep 18 15:58:03.244154 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.244098    2585 scope.go:117] "RemoveContainer" containerID="518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5"
Sep 18 15:58:03.244339 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:58:03.244323    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5\": container with ID starting with 518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5 not found: ID does not exist" containerID="518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5"
Sep 18 15:58:03.244429 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.244343    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5"} err="failed to get container status \"518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5\": rpc error: code = NotFound desc = could not find container \"518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5\": container with ID starting with 518ef962dc3ccc430160ba8b0d765c627c3c8714553581d0564a7734181fd3a5 not found: ID does not exist"
Sep 18 15:58:03.255715 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.255695    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"]
Sep 18 15:58:03.261104 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.261080    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-998130-openshift-default-5f678668d8-vzs6t"]
Sep 18 15:58:03.524295 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:03.524257    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" path="/var/lib/kubelet/pods/30764f3a-5b5f-4122-a3be-126ce108115f/volumes"
Sep 18 15:58:06.894723 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:06.894692    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"]
Sep 18 15:58:06.895199 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:06.894988    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy" containerID="cri-o://a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278" gracePeriod=30
Sep 18 15:58:11.365107 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:11.365070    2585 patch_prober.go:28] interesting pod/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 15:58:11.365513 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:11.365136    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 15:58:12.156106 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.156082    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:58:12.243476 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243450    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istiod-ca-cert\") pod \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") "
Sep 18 15:58:12.243609 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243487    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-data\") pod \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") "
Sep 18 15:58:12.243609 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243526    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-certs\") pod \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") "
Sep 18 15:58:12.243704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243637    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-envoy\") pod \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") "
Sep 18 15:58:12.243704 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243684    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-credential-socket\") pod \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") "
Sep 18 15:58:12.243790 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243708    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-587fg\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-kube-api-access-587fg\") pod \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") "
Sep 18 15:58:12.243790 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243758    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-socket\") pod \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") "
Sep 18 15:58:12.243790 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243770    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e62f7b7-36bb-48e2-98b3-98ba655f51d8" (UID: "3e62f7b7-36bb-48e2-98b3-98ba655f51d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:12.243790 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243786    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-token\") pod \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") "
Sep 18 15:58:12.244023 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243864    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-podinfo\") pod \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\" (UID: \"3e62f7b7-36bb-48e2-98b3-98ba655f51d8\") "
Sep 18 15:58:12.244023 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.243785    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e62f7b7-36bb-48e2-98b3-98ba655f51d8" (UID: "3e62f7b7-36bb-48e2-98b3-98ba655f51d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:12.244134 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.244035    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e62f7b7-36bb-48e2-98b3-98ba655f51d8" (UID: "3e62f7b7-36bb-48e2-98b3-98ba655f51d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:12.244134 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.244071    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e62f7b7-36bb-48e2-98b3-98ba655f51d8" (UID: "3e62f7b7-36bb-48e2-98b3-98ba655f51d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 15:58:12.244260 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.244239    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:12.244319 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.244272    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:12.244319 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.244289    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:12.244319 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.244302    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:12.244319 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.244240    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e62f7b7-36bb-48e2-98b3-98ba655f51d8" (UID: "3e62f7b7-36bb-48e2-98b3-98ba655f51d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:12.245865 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.245812    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e62f7b7-36bb-48e2-98b3-98ba655f51d8" (UID: "3e62f7b7-36bb-48e2-98b3-98ba655f51d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 15:58:12.245865 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.245858    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-kube-api-access-587fg" (OuterVolumeSpecName: "kube-api-access-587fg") pod "3e62f7b7-36bb-48e2-98b3-98ba655f51d8" (UID: "3e62f7b7-36bb-48e2-98b3-98ba655f51d8"). InnerVolumeSpecName "kube-api-access-587fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:58:12.246199 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.246181    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e62f7b7-36bb-48e2-98b3-98ba655f51d8" (UID: "3e62f7b7-36bb-48e2-98b3-98ba655f51d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:58:12.246245 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.246202    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e62f7b7-36bb-48e2-98b3-98ba655f51d8" (UID: "3e62f7b7-36bb-48e2-98b3-98ba655f51d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 15:58:12.278052 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.278025    2585 generic.go:356] "Generic (PLEG): container finished" podID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerID="a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278" exitCode=0
Sep 18 15:58:12.278158 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.278079    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"
Sep 18 15:58:12.278158 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.278091    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" event={"ID":"3e62f7b7-36bb-48e2-98b3-98ba655f51d8","Type":"ContainerDied","Data":"a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278"}
Sep 18 15:58:12.278158 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.278145    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv" event={"ID":"3e62f7b7-36bb-48e2-98b3-98ba655f51d8","Type":"ContainerDied","Data":"fb2cf7761a48b5d1f5b76af9d0f15d24db5c0ccbe0c21ffc97146bbe32554ab9"}
Sep 18 15:58:12.278260 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.278175    2585 scope.go:117] "RemoveContainer" containerID="a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278"
Sep 18 15:58:12.294959 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.294943    2585 scope.go:117] "RemoveContainer" containerID="31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e"
Sep 18 15:58:12.303238 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.303219    2585 scope.go:117] "RemoveContainer" containerID="a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278"
Sep 18 15:58:12.303733 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:58:12.303712    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278\": container with ID starting with a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278 not found: ID does not exist" containerID="a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278"
Sep 18 15:58:12.303832 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.303743    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278"} err="failed to get container status \"a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278\": rpc error: code = NotFound desc = could not find container \"a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278\": container with ID starting with a3732314c96c2d0217098719b54fdbd47cdb32c1680d0aff370264fbde98f278 not found: ID does not exist"
Sep 18 15:58:12.303832 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.303766    2585 scope.go:117] "RemoveContainer" containerID="31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e"
Sep 18 15:58:12.304073 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:58:12.304055    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e\": container with ID starting with 31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e not found: ID does not exist" containerID="31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e"
Sep 18 15:58:12.304124 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.304078    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e"} err="failed to get container status \"31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e\": rpc error: code = NotFound desc = could not find container \"31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e\": container with ID starting with 31e87922846212ab0918e22684aafee2fc74d2d35406d2d28de5fb2becebed7e not found: ID does not exist"
Sep 18 15:58:12.305765 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.305743    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"]
Sep 18 15:58:12.312695 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.312671    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c56bfef6-openshift-default-77d478d75b-gbchv"]
Sep 18 15:58:12.345709 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.345687    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:12.345709 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.345708    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:12.345831 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.345717    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-587fg\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-kube-api-access-587fg\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:12.345831 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.345726    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:12.345831 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:12.345734    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e62f7b7-36bb-48e2-98b3-98ba655f51d8-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:13.523688 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:13.523653    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" path="/var/lib/kubelet/pods/3e62f7b7-36bb-48e2-98b3-98ba655f51d8/volumes"
Sep 18 15:58:50.931505 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.931470    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9134-b7737b"]
Sep 18 15:58:50.932093 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932048    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828f6c98-b645-4fff-bf28-e4aa47ef5f3e" containerName="main"
Sep 18 15:58:50.932093 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932069    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="828f6c98-b645-4fff-bf28-e4aa47ef5f3e" containerName="main"
Sep 18 15:58:50.932093 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932084    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy"
Sep 18 15:58:50.932093 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932093    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932108    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932116    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932128    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a71489e6-0651-4a4e-880e-dde1041985f8" containerName="maas-api"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932136    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a71489e6-0651-4a4e-880e-dde1041985f8" containerName="maas-api"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932144    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0672ce54-1368-489c-a172-0160ead37f0f" containerName="main"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932152    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0672ce54-1368-489c-a172-0160ead37f0f" containerName="main"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932175    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ddd6583-997b-4500-a2fe-5fe32e9362c9" containerName="maas-api"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932183    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ddd6583-997b-4500-a2fe-5fe32e9362c9" containerName="maas-api"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932199    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932207    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932223    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932231    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932241    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbb8dd38-5ae9-459b-9e13-e4b3de039143" containerName="maas-api"
Sep 18 15:58:50.932323 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932248    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbb8dd38-5ae9-459b-9e13-e4b3de039143" containerName="maas-api"
Sep 18 15:58:50.933007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932337    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbb8dd38-5ae9-459b-9e13-e4b3de039143" containerName="maas-api"
Sep 18 15:58:50.933007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932350    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="828f6c98-b645-4fff-bf28-e4aa47ef5f3e" containerName="main"
Sep 18 15:58:50.933007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932360    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0672ce54-1368-489c-a172-0160ead37f0f" containerName="main"
Sep 18 15:58:50.933007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932371    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy"
Sep 18 15:58:50.933007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932380    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ddd6583-997b-4500-a2fe-5fe32e9362c9" containerName="maas-api"
Sep 18 15:58:50.933007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932390    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy"
Sep 18 15:58:50.933007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932399    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a71489e6-0651-4a4e-880e-dde1041985f8" containerName="maas-api"
Sep 18 15:58:50.933007 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.932407    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e62f7b7-36bb-48e2-98b3-98ba655f51d8" containerName="istio-proxy"
Sep 18 15:58:50.937166 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.937146    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-b7737b"
Sep 18 15:58:50.939682 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.939661    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w9zf6\""
Sep 18 15:58:50.941735 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:50.941710    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9134-b7737b"]
Sep 18 15:58:51.008441 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:51.008405    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85rs8\" (UniqueName: \"kubernetes.io/projected/df9cb1ab-55e0-48b0-8547-fcebe52c74b8-kube-api-access-85rs8\") pod \"test-curl-9134-b7737b\" (UID: \"df9cb1ab-55e0-48b0-8547-fcebe52c74b8\") " pod="opendatahub/test-curl-9134-b7737b"
Sep 18 15:58:51.109763 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:51.109723    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rs8\" (UniqueName: \"kubernetes.io/projected/df9cb1ab-55e0-48b0-8547-fcebe52c74b8-kube-api-access-85rs8\") pod \"test-curl-9134-b7737b\" (UID: \"df9cb1ab-55e0-48b0-8547-fcebe52c74b8\") " pod="opendatahub/test-curl-9134-b7737b"
Sep 18 15:58:51.118483 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:51.118457    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85rs8\" (UniqueName: \"kubernetes.io/projected/df9cb1ab-55e0-48b0-8547-fcebe52c74b8-kube-api-access-85rs8\") pod \"test-curl-9134-b7737b\" (UID: \"df9cb1ab-55e0-48b0-8547-fcebe52c74b8\") " pod="opendatahub/test-curl-9134-b7737b"
Sep 18 15:58:51.247742 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:51.247713    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-b7737b"
Sep 18 15:58:51.521638 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:58:51.521607    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf9cb1ab_55e0_48b0_8547_fcebe52c74b8.slice/crio-a83b4ec01b97a82d31005c614a70fea18b1d8a3554b947fa16cbd959184a5293 WatchSource:0}: Error finding container a83b4ec01b97a82d31005c614a70fea18b1d8a3554b947fa16cbd959184a5293: Status 404 returned error can't find the container with id a83b4ec01b97a82d31005c614a70fea18b1d8a3554b947fa16cbd959184a5293
Sep 18 15:58:51.523185 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:51.523160    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 15:58:51.525395 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:51.525373    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9134-b7737b"]
Sep 18 15:58:52.520797 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:52.520755    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9134-b7737b" event={"ID":"df9cb1ab-55e0-48b0-8547-fcebe52c74b8","Type":"ContainerStarted","Data":"e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20"}
Sep 18 15:58:52.521184 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:52.520810    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9134-b7737b" event={"ID":"df9cb1ab-55e0-48b0-8547-fcebe52c74b8","Type":"ContainerStarted","Data":"a83b4ec01b97a82d31005c614a70fea18b1d8a3554b947fa16cbd959184a5293"}
Sep 18 15:58:52.536145 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:52.536101    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9134-b7737b" podStartSLOduration=1.960156902 podStartE2EDuration="2.536086457s" totalImagesPullingTime="575.929555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:58:52.534727163 +0000 UTC m=+1457.193519827" watchObservedRunningTime="2026-09-18 15:58:52.536086457 +0000 UTC m=+1457.194879160"
Sep 18 15:58:53.145005 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:53.144971    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9134-b7737b"]
Sep 18 15:58:53.154085 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:53.154057    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9134-b7737b"]
Sep 18 15:58:53.528255 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:58:53.528217    2585 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9134-b7737b" container="test-curl-9134-b7737b"
Sep 18 15:58:54.534583 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:54.534546    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9134-b7737b" podUID="df9cb1ab-55e0-48b0-8547-fcebe52c74b8" containerName="test-curl-9134-b7737b" containerID="cri-o://e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20" gracePeriod=2
Sep 18 15:58:56.716806 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:56.716780    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-b7737b"
Sep 18 15:58:56.761615 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:56.761582    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85rs8\" (UniqueName: \"kubernetes.io/projected/df9cb1ab-55e0-48b0-8547-fcebe52c74b8-kube-api-access-85rs8\") pod \"df9cb1ab-55e0-48b0-8547-fcebe52c74b8\" (UID: \"df9cb1ab-55e0-48b0-8547-fcebe52c74b8\") "
Sep 18 15:58:56.763730 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:56.763705    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df9cb1ab-55e0-48b0-8547-fcebe52c74b8-kube-api-access-85rs8" (OuterVolumeSpecName: "kube-api-access-85rs8") pod "df9cb1ab-55e0-48b0-8547-fcebe52c74b8" (UID: "df9cb1ab-55e0-48b0-8547-fcebe52c74b8"). InnerVolumeSpecName "kube-api-access-85rs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 15:58:56.862388 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:56.862324    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85rs8\" (UniqueName: \"kubernetes.io/projected/df9cb1ab-55e0-48b0-8547-fcebe52c74b8-kube-api-access-85rs8\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 15:58:57.524512 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:57.524471    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df9cb1ab-55e0-48b0-8547-fcebe52c74b8" path="/var/lib/kubelet/pods/df9cb1ab-55e0-48b0-8547-fcebe52c74b8/volumes"
Sep 18 15:58:57.552266 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:57.552233    2585 generic.go:356] "Generic (PLEG): container finished" podID="df9cb1ab-55e0-48b0-8547-fcebe52c74b8" containerID="e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20" exitCode=137
Sep 18 15:58:57.552368 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:57.552270    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-b7737b"
Sep 18 15:58:57.552368 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:57.552314    2585 scope.go:117] "RemoveContainer" containerID="e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20"
Sep 18 15:58:57.561004 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:57.560988    2585 scope.go:117] "RemoveContainer" containerID="e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20"
Sep 18 15:58:57.561295 ip-10-0-130-208 kubenswrapper[2585]: E0918 15:58:57.561266    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20\": container with ID starting with e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20 not found: ID does not exist" containerID="e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20"
Sep 18 15:58:57.561376 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:58:57.561302    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20"} err="failed to get container status \"e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20\": rpc error: code = NotFound desc = could not find container \"e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20\": container with ID starting with e9895b0229366757ac24fb28495eb93b6f40961f76595db6b0e4d93914784f20 not found: ID does not exist"
Sep 18 15:59:03.244724 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.244684    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"]
Sep 18 15:59:03.245150 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.245133    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df9cb1ab-55e0-48b0-8547-fcebe52c74b8" containerName="test-curl-9134-b7737b"
Sep 18 15:59:03.245209 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.245152    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="df9cb1ab-55e0-48b0-8547-fcebe52c74b8" containerName="test-curl-9134-b7737b"
Sep 18 15:59:03.245248 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.245223    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="df9cb1ab-55e0-48b0-8547-fcebe52c74b8" containerName="test-curl-9134-b7737b"
Sep 18 15:59:03.245248 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.245233    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="30764f3a-5b5f-4122-a3be-126ce108115f" containerName="istio-proxy"
Sep 18 15:59:03.249829 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.249807    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.257281 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.257259    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-769e22-openshift-default-dockercfg-5hxd5\""
Sep 18 15:59:03.267215 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.267189    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"]
Sep 18 15:59:03.321880 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.321810    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-podinfo\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.322057 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.321896    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-data\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.322057 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.321959    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-socket\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.322057 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.322016    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-credential-socket\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.322057 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.322051    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-envoy\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.322226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.322080    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-token\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.322226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.322138    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzknx\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-kube-api-access-fzknx\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.322226 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.322177    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-certs\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.322326 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.322276    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istiod-ca-cert\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.422743 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.422712    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istiod-ca-cert\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.422956 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.422754    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-podinfo\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.422956 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.422777    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-data\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.422956 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.422807    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-socket\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.423132 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.422951    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-credential-socket\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.423132 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.422996    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-envoy\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.423132 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.423029    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-token\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.423132 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.423094    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzknx\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-kube-api-access-fzknx\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.423322 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.423139    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-certs\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.423377 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.423330    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-data\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.423613 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.423577    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istiod-ca-cert\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.423803 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.423590    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-credential-socket\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.423803 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.423688    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-socket\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.424012 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.423737    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-certs\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.425549 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.425525    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-envoy\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.425549 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.425539    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-podinfo\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.433435 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.433412    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-token\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.433705 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.433683    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzknx\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-kube-api-access-fzknx\") pod \"e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") " pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.560012 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.559929    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:03.845557 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.845529    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"]
Sep 18 15:59:03.846307 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:59:03.846286    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc1d3388_4bec_44e2_b14e_b8bc5d70f465.slice/crio-1e95e6596f6327d9ac991da2a8f536a1e72346129cff2a62ead2d46cd07fe33c WatchSource:0}: Error finding container 1e95e6596f6327d9ac991da2a8f536a1e72346129cff2a62ead2d46cd07fe33c: Status 404 returned error can't find the container with id 1e95e6596f6327d9ac991da2a8f536a1e72346129cff2a62ead2d46cd07fe33c
Sep 18 15:59:03.848819 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.848780    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:59:03.848923 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.848891    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:59:03.848968 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:03.848941    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:59:04.604443 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:04.604411    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" event={"ID":"bc1d3388-4bec-44e2-b14e-b8bc5d70f465","Type":"ContainerStarted","Data":"fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09"}
Sep 18 15:59:04.604802 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:04.604487    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" event={"ID":"bc1d3388-4bec-44e2-b14e-b8bc5d70f465","Type":"ContainerStarted","Data":"1e95e6596f6327d9ac991da2a8f536a1e72346129cff2a62ead2d46cd07fe33c"}
Sep 18 15:59:04.642297 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:04.642253    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" podStartSLOduration=1.6422388479999999 podStartE2EDuration="1.642238848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:59:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:59:04.640129923 +0000 UTC m=+1469.298922583" watchObservedRunningTime="2026-09-18 15:59:04.642238848 +0000 UTC m=+1469.301031507"
Sep 18 15:59:05.560409 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:05.560376    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:05.565592 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:05.565570    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:05.609773 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:05.609737    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:05.610470 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:05.610450    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:10.259670 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.259636    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"]
Sep 18 15:59:10.270714 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.270680    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.274298 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.274268    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-769e22-serving-cert\""
Sep 18 15:59:10.278339 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.278309    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"]
Sep 18 15:59:10.392089 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.392053    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-metrics-certs\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.392089 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.392095    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-maas-api-tls\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.392331 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.392195    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2krd\" (UniqueName: \"kubernetes.io/projected/7a142de8-cd56-433b-aa7a-397c5f354167-kube-api-access-f2krd\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.392331 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.392317    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a142de8-cd56-433b-aa7a-397c5f354167-openshift-service-ca\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.493580 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.493547    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a142de8-cd56-433b-aa7a-397c5f354167-openshift-service-ca\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.493784 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.493611    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-metrics-certs\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.493784 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.493647    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-maas-api-tls\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.493784 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.493711    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2krd\" (UniqueName: \"kubernetes.io/projected/7a142de8-cd56-433b-aa7a-397c5f354167-kube-api-access-f2krd\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.494341 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.494319    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a142de8-cd56-433b-aa7a-397c5f354167-openshift-service-ca\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.496090 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.496064    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-maas-api-tls\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.496189 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.496109    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-metrics-certs\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.501998 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.501977    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2krd\" (UniqueName: \"kubernetes.io/projected/7a142de8-cd56-433b-aa7a-397c5f354167-kube-api-access-f2krd\") pod \"maas-api-e2e-api-a-769e22-74d666767-88w49\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.584645 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.584556    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:10.869484 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:10.869446    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"]
Sep 18 15:59:10.870776 ip-10-0-130-208 kubenswrapper[2585]: W0918 15:59:10.870747    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a142de8_cd56_433b_aa7a_397c5f354167.slice/crio-34fc06aebdc8898b972bcbb939b5f008491175b8a79ab3ab7bceb4a4477f9473 WatchSource:0}: Error finding container 34fc06aebdc8898b972bcbb939b5f008491175b8a79ab3ab7bceb4a4477f9473: Status 404 returned error can't find the container with id 34fc06aebdc8898b972bcbb939b5f008491175b8a79ab3ab7bceb4a4477f9473
Sep 18 15:59:11.649602 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:11.649561    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49" event={"ID":"7a142de8-cd56-433b-aa7a-397c5f354167","Type":"ContainerStarted","Data":"049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da"}
Sep 18 15:59:11.650145 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:11.649624    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49" event={"ID":"7a142de8-cd56-433b-aa7a-397c5f354167","Type":"ContainerStarted","Data":"34fc06aebdc8898b972bcbb939b5f008491175b8a79ab3ab7bceb4a4477f9473"}
Sep 18 15:59:11.650145 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:11.649650    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:11.671632 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:11.671577    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49" podStartSLOduration=1.32022112 podStartE2EDuration="1.671562651s" totalImagesPullingTime="351.341531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 15:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 15:59:11.668866997 +0000 UTC m=+1476.327659652" watchObservedRunningTime="2026-09-18 15:59:11.671562651 +0000 UTC m=+1476.330355311"
Sep 18 15:59:17.660991 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:17.660961    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 15:59:25.733431 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:25.733398    2585 generic.go:356] "Generic (PLEG): container finished" podID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerID="fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09" exitCode=0
Sep 18 15:59:25.733961 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:25.733480    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" event={"ID":"bc1d3388-4bec-44e2-b14e-b8bc5d70f465","Type":"ContainerDied","Data":"fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09"}
Sep 18 15:59:25.734072 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:25.734054    2585 scope.go:117] "RemoveContainer" containerID="fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09"
Sep 18 15:59:25.735287 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:25.735262    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:59:25.735362 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:25.735330    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:59:25.735362 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:25.735359    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 15:59:26.742254 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:26.742210    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" event={"ID":"bc1d3388-4bec-44e2-b14e-b8bc5d70f465","Type":"ContainerStarted","Data":"6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c"}
Sep 18 15:59:27.560227 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:27.560193    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:27.560446 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:27.560425    2585 patch_prober.go:28] interesting pod/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 15:59:27.560539 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:27.560493    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 15:59:28.560674 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:28.560638    2585 patch_prober.go:28] interesting pod/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 15:59:28.561089 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:28.560708    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 15:59:29.560386 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:29.560346    2585 patch_prober.go:28] interesting pod/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 15:59:29.560584 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:29.560424    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 15:59:30.560493 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:30.560460    2585 patch_prober.go:28] interesting pod/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 15:59:30.561018 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:30.560535    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 15:59:31.561060 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:31.561022    2585 patch_prober.go:28] interesting pod/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 15:59:31.561514 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:31.561083    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 15:59:32.564807 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:32.564777    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:32.565294 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:32.565178    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:32.565786 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:32.565767    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 15:59:37.554204 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:37.554178    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 15:59:37.561932 ip-10-0-130-208 kubenswrapper[2585]: I0918 15:59:37.560263    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 16:00:00.200551 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:00.200470    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p"]
Sep 18 16:00:00.203656 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:00.203635    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p"
Sep 18 16:00:00.207873 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:00.207834    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hg46\""
Sep 18 16:00:00.214526 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:00.214504    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p"
Sep 18 16:00:00.229559 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:00.229343    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p"]
Sep 18 16:00:00.497885 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:00.497854    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p"]
Sep 18 16:00:00.499344 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:00:00.499313    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddec6bdf9_4a8b_49ff_b703_8261497b87fa.slice/crio-8693ef9c9714419cba213d1d4b63c3b0d6fdface86225231c4982e4459add40c WatchSource:0}: Error finding container 8693ef9c9714419cba213d1d4b63c3b0d6fdface86225231c4982e4459add40c: Status 404 returned error can't find the container with id 8693ef9c9714419cba213d1d4b63c3b0d6fdface86225231c4982e4459add40c
Sep 18 16:00:00.944117 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:00.944032    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p" event={"ID":"dec6bdf9-4a8b-49ff-b703-8261497b87fa","Type":"ContainerStarted","Data":"8693ef9c9714419cba213d1d4b63c3b0d6fdface86225231c4982e4459add40c"}
Sep 18 16:00:03.965416 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:03.965386    2585 generic.go:356] "Generic (PLEG): container finished" podID="dec6bdf9-4a8b-49ff-b703-8261497b87fa" containerID="812d8535c02060937fe07676deaff402581e7e47b1f2bf70f06e8148751d687c" exitCode=0
Sep 18 16:00:03.965782 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:03.965473    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p" event={"ID":"dec6bdf9-4a8b-49ff-b703-8261497b87fa","Type":"ContainerDied","Data":"812d8535c02060937fe07676deaff402581e7e47b1f2bf70f06e8148751d687c"}
Sep 18 16:00:05.093134 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:05.093110    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p"
Sep 18 16:00:05.978422 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:05.978344    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p"
Sep 18 16:00:05.978422 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:05.978363    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p" event={"ID":"dec6bdf9-4a8b-49ff-b703-8261497b87fa","Type":"ContainerDied","Data":"8693ef9c9714419cba213d1d4b63c3b0d6fdface86225231c4982e4459add40c"}
Sep 18 16:00:05.978644 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:05.978437    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8693ef9c9714419cba213d1d4b63c3b0d6fdface86225231c4982e4459add40c"
Sep 18 16:00:29.085375 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.085339    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"]
Sep 18 16:00:29.086046 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.085598    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49" podUID="7a142de8-cd56-433b-aa7a-397c5f354167" containerName="maas-api" containerID="cri-o://049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da" gracePeriod=30
Sep 18 16:00:29.348945 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.348859    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 16:00:29.359716 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.359643    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-maas-api-tls\") pod \"7a142de8-cd56-433b-aa7a-397c5f354167\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") "
Sep 18 16:00:29.363011 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.362860    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a142de8-cd56-433b-aa7a-397c5f354167-openshift-service-ca\") pod \"7a142de8-cd56-433b-aa7a-397c5f354167\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") "
Sep 18 16:00:29.363011 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.362938    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-metrics-certs\") pod \"7a142de8-cd56-433b-aa7a-397c5f354167\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") "
Sep 18 16:00:29.363490 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.363321    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2krd\" (UniqueName: \"kubernetes.io/projected/7a142de8-cd56-433b-aa7a-397c5f354167-kube-api-access-f2krd\") pod \"7a142de8-cd56-433b-aa7a-397c5f354167\" (UID: \"7a142de8-cd56-433b-aa7a-397c5f354167\") "
Sep 18 16:00:29.364514 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.364471    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a142de8-cd56-433b-aa7a-397c5f354167-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a142de8-cd56-433b-aa7a-397c5f354167" (UID: "7a142de8-cd56-433b-aa7a-397c5f354167"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:00:29.374706 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.373207    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a142de8-cd56-433b-aa7a-397c5f354167" (UID: "7a142de8-cd56-433b-aa7a-397c5f354167"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:00:29.375974 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.375934    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7a142de8-cd56-433b-aa7a-397c5f354167" (UID: "7a142de8-cd56-433b-aa7a-397c5f354167"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:00:29.381855 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.381799    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a142de8-cd56-433b-aa7a-397c5f354167-kube-api-access-f2krd" (OuterVolumeSpecName: "kube-api-access-f2krd") pod "7a142de8-cd56-433b-aa7a-397c5f354167" (UID: "7a142de8-cd56-433b-aa7a-397c5f354167"). InnerVolumeSpecName "kube-api-access-f2krd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:00:29.465116 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.465084    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:29.465116 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.465112    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a142de8-cd56-433b-aa7a-397c5f354167-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:29.465116 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.465121    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a142de8-cd56-433b-aa7a-397c5f354167-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:29.465332 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:29.465129    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2krd\" (UniqueName: \"kubernetes.io/projected/7a142de8-cd56-433b-aa7a-397c5f354167-kube-api-access-f2krd\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:30.122089 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:30.122058    2585 generic.go:356] "Generic (PLEG): container finished" podID="7a142de8-cd56-433b-aa7a-397c5f354167" containerID="049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da" exitCode=0
Sep 18 16:00:30.122522 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:30.122107    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"
Sep 18 16:00:30.122522 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:30.122121    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49" event={"ID":"7a142de8-cd56-433b-aa7a-397c5f354167","Type":"ContainerDied","Data":"049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da"}
Sep 18 16:00:30.122522 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:30.122178    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49" event={"ID":"7a142de8-cd56-433b-aa7a-397c5f354167","Type":"ContainerDied","Data":"34fc06aebdc8898b972bcbb939b5f008491175b8a79ab3ab7bceb4a4477f9473"}
Sep 18 16:00:30.122522 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:30.122203    2585 scope.go:117] "RemoveContainer" containerID="049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da"
Sep 18 16:00:30.131665 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:30.131646    2585 scope.go:117] "RemoveContainer" containerID="049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da"
Sep 18 16:00:30.131965 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:00:30.131942    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da\": container with ID starting with 049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da not found: ID does not exist" containerID="049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da"
Sep 18 16:00:30.132026 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:30.131974    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da"} err="failed to get container status \"049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da\": rpc error: code = NotFound desc = could not find container \"049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da\": container with ID starting with 049583f2915665127434ec4d5e545f382dfca035e13ae53e16f91334307235da not found: ID does not exist"
Sep 18 16:00:30.145778 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:30.145754    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"]
Sep 18 16:00:30.150737 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:30.150718    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-769e22-74d666767-88w49"]
Sep 18 16:00:31.524854 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:31.524811    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a142de8-cd56-433b-aa7a-397c5f354167" path="/var/lib/kubelet/pods/7a142de8-cd56-433b-aa7a-397c5f354167/volumes"
Sep 18 16:00:36.430629 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:36.430595    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"]
Sep 18 16:00:36.431110 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:36.430913    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy" containerID="cri-o://6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c" gracePeriod=30
Sep 18 16:00:41.682347 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.682318    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 16:00:41.788654 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788624    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-data\") pod \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") "
Sep 18 16:00:41.788654 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788657    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-credential-socket\") pod \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") "
Sep 18 16:00:41.788952 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788683    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-envoy\") pod \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") "
Sep 18 16:00:41.788952 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788713    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-certs\") pod \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") "
Sep 18 16:00:41.788952 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788732    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istiod-ca-cert\") pod \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") "
Sep 18 16:00:41.788952 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788781    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-socket\") pod \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") "
Sep 18 16:00:41.788952 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788826    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-token\") pod \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") "
Sep 18 16:00:41.788952 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788893    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-podinfo\") pod \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") "
Sep 18 16:00:41.788952 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788931    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzknx\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-kube-api-access-fzknx\") pod \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\" (UID: \"bc1d3388-4bec-44e2-b14e-b8bc5d70f465\") "
Sep 18 16:00:41.789303 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.788975    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-data" (OuterVolumeSpecName: "istio-data") pod "bc1d3388-4bec-44e2-b14e-b8bc5d70f465" (UID: "bc1d3388-4bec-44e2-b14e-b8bc5d70f465"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:00:41.789303 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.789009    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bc1d3388-4bec-44e2-b14e-b8bc5d70f465" (UID: "bc1d3388-4bec-44e2-b14e-b8bc5d70f465"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:00:41.789303 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.789062    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bc1d3388-4bec-44e2-b14e-b8bc5d70f465" (UID: "bc1d3388-4bec-44e2-b14e-b8bc5d70f465"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:00:41.789303 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.789162    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bc1d3388-4bec-44e2-b14e-b8bc5d70f465" (UID: "bc1d3388-4bec-44e2-b14e-b8bc5d70f465"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:00:41.789541 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.789363    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bc1d3388-4bec-44e2-b14e-b8bc5d70f465" (UID: "bc1d3388-4bec-44e2-b14e-b8bc5d70f465"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:00:41.789541 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.789371    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:41.789541 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.789407    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:41.789541 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.789422    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:41.789541 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.789437    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:41.791089 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.791063    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bc1d3388-4bec-44e2-b14e-b8bc5d70f465" (UID: "bc1d3388-4bec-44e2-b14e-b8bc5d70f465"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 16:00:41.791197 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.791144    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-kube-api-access-fzknx" (OuterVolumeSpecName: "kube-api-access-fzknx") pod "bc1d3388-4bec-44e2-b14e-b8bc5d70f465" (UID: "bc1d3388-4bec-44e2-b14e-b8bc5d70f465"). InnerVolumeSpecName "kube-api-access-fzknx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:00:41.791296 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.791274    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-token" (OuterVolumeSpecName: "istio-token") pod "bc1d3388-4bec-44e2-b14e-b8bc5d70f465" (UID: "bc1d3388-4bec-44e2-b14e-b8bc5d70f465"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:00:41.791352 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.791311    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bc1d3388-4bec-44e2-b14e-b8bc5d70f465" (UID: "bc1d3388-4bec-44e2-b14e-b8bc5d70f465"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:00:41.890610 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.890584    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:41.890610 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.890611    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:41.890790 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.890624    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzknx\" (UniqueName: \"kubernetes.io/projected/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-kube-api-access-fzknx\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:41.890790 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.890636    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:41.890790 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:41.890648    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1d3388-4bec-44e2-b14e-b8bc5d70f465-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:42.194278 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.194189    2585 generic.go:356] "Generic (PLEG): container finished" podID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerID="6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c" exitCode=0
Sep 18 16:00:42.194278 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.194236    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" event={"ID":"bc1d3388-4bec-44e2-b14e-b8bc5d70f465","Type":"ContainerDied","Data":"6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c"}
Sep 18 16:00:42.194278 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.194275    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m" event={"ID":"bc1d3388-4bec-44e2-b14e-b8bc5d70f465","Type":"ContainerDied","Data":"1e95e6596f6327d9ac991da2a8f536a1e72346129cff2a62ead2d46cd07fe33c"}
Sep 18 16:00:42.194513 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.194295    2585 scope.go:117] "RemoveContainer" containerID="6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c"
Sep 18 16:00:42.194513 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.194239    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"
Sep 18 16:00:42.204463 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.204447    2585 scope.go:117] "RemoveContainer" containerID="fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09"
Sep 18 16:00:42.212866 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.212831    2585 scope.go:117] "RemoveContainer" containerID="6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c"
Sep 18 16:00:42.213160 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:00:42.213134    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c\": container with ID starting with 6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c not found: ID does not exist" containerID="6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c"
Sep 18 16:00:42.213253 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.213172    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c"} err="failed to get container status \"6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c\": rpc error: code = NotFound desc = could not find container \"6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c\": container with ID starting with 6bda37d7db84aefc6be362caa64d5b854802a18de0f0e8a0214acc4e421e0a5c not found: ID does not exist"
Sep 18 16:00:42.213253 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.213196    2585 scope.go:117] "RemoveContainer" containerID="fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09"
Sep 18 16:00:42.213489 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:00:42.213455    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09\": container with ID starting with fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09 not found: ID does not exist" containerID="fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09"
Sep 18 16:00:42.213581 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.213496    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09"} err="failed to get container status \"fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09\": rpc error: code = NotFound desc = could not find container \"fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09\": container with ID starting with fe0cfab61ca74b7d8f6581b56e5ace3b5382893998c462166a09d1fca23a4f09 not found: ID does not exist"
Sep 18 16:00:42.218701 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.218677    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"]
Sep 18 16:00:42.223751 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:42.223673    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-769e22-openshift-default-849f94c68b-m8g6m"]
Sep 18 16:00:43.523531 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:43.523496    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" path="/var/lib/kubelet/pods/bc1d3388-4bec-44e2-b14e-b8bc5d70f465/volumes"
Sep 18 16:00:47.458419 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458387    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9134-98334b"]
Sep 18 16:00:47.458784 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458767    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dec6bdf9-4a8b-49ff-b703-8261497b87fa" containerName="cleanup"
Sep 18 16:00:47.458784 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458779    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dec6bdf9-4a8b-49ff-b703-8261497b87fa" containerName="cleanup"
Sep 18 16:00:47.458895 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458788    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy"
Sep 18 16:00:47.458895 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458794    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy"
Sep 18 16:00:47.458895 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458802    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy"
Sep 18 16:00:47.458895 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458807    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy"
Sep 18 16:00:47.458895 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458818    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a142de8-cd56-433b-aa7a-397c5f354167" containerName="maas-api"
Sep 18 16:00:47.458895 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458823    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a142de8-cd56-433b-aa7a-397c5f354167" containerName="maas-api"
Sep 18 16:00:47.459080 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458921    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy"
Sep 18 16:00:47.459080 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458930    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dec6bdf9-4a8b-49ff-b703-8261497b87fa" containerName="cleanup"
Sep 18 16:00:47.459080 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.458938    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a142de8-cd56-433b-aa7a-397c5f354167" containerName="maas-api"
Sep 18 16:00:47.462458 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.462441    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-98334b"
Sep 18 16:00:47.465444 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.465419    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w9zf6\""
Sep 18 16:00:47.469216 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.469194    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9134-98334b"]
Sep 18 16:00:47.548914 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.548885    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh9tn\" (UniqueName: \"kubernetes.io/projected/785b2dc3-8249-46d9-9b0d-2d751e4c6434-kube-api-access-sh9tn\") pod \"test-curl-9134-98334b\" (UID: \"785b2dc3-8249-46d9-9b0d-2d751e4c6434\") " pod="opendatahub/test-curl-9134-98334b"
Sep 18 16:00:47.649966 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.649935    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9tn\" (UniqueName: \"kubernetes.io/projected/785b2dc3-8249-46d9-9b0d-2d751e4c6434-kube-api-access-sh9tn\") pod \"test-curl-9134-98334b\" (UID: \"785b2dc3-8249-46d9-9b0d-2d751e4c6434\") " pod="opendatahub/test-curl-9134-98334b"
Sep 18 16:00:47.658393 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.658368    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh9tn\" (UniqueName: \"kubernetes.io/projected/785b2dc3-8249-46d9-9b0d-2d751e4c6434-kube-api-access-sh9tn\") pod \"test-curl-9134-98334b\" (UID: \"785b2dc3-8249-46d9-9b0d-2d751e4c6434\") " pod="opendatahub/test-curl-9134-98334b"
Sep 18 16:00:47.774160 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:47.774135    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-98334b"
Sep 18 16:00:48.073456 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:48.073430    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9134-98334b"]
Sep 18 16:00:48.075637 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:00:48.075613    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod785b2dc3_8249_46d9_9b0d_2d751e4c6434.slice/crio-3ed7bc5f5297107df3308fb2bedb32ff5cf7cc6d511a463d8c9f105f2d6eeec9 WatchSource:0}: Error finding container 3ed7bc5f5297107df3308fb2bedb32ff5cf7cc6d511a463d8c9f105f2d6eeec9: Status 404 returned error can't find the container with id 3ed7bc5f5297107df3308fb2bedb32ff5cf7cc6d511a463d8c9f105f2d6eeec9
Sep 18 16:00:48.234137 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:48.234095    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9134-98334b" event={"ID":"785b2dc3-8249-46d9-9b0d-2d751e4c6434","Type":"ContainerStarted","Data":"28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a"}
Sep 18 16:00:48.234305 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:48.234149    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9134-98334b" event={"ID":"785b2dc3-8249-46d9-9b0d-2d751e4c6434","Type":"ContainerStarted","Data":"3ed7bc5f5297107df3308fb2bedb32ff5cf7cc6d511a463d8c9f105f2d6eeec9"}
Sep 18 16:00:48.257054 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:48.257001    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9134-98334b" podStartSLOduration=1.256986413 podStartE2EDuration="1.256986413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:00:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:00:48.256376408 +0000 UTC m=+1572.915169067" watchObservedRunningTime="2026-09-18 16:00:48.256986413 +0000 UTC m=+1572.915779095"
Sep 18 16:00:49.050801 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.050705    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9134-98334b"]
Sep 18 16:00:49.053336 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.053309    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9134-98334b"]
Sep 18 16:00:49.224059 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.224025    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9134-38e005"]
Sep 18 16:00:49.224406 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.224392    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="785b2dc3-8249-46d9-9b0d-2d751e4c6434" containerName="test-curl-9134-98334b"
Sep 18 16:00:49.224461 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.224407    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="785b2dc3-8249-46d9-9b0d-2d751e4c6434" containerName="test-curl-9134-98334b"
Sep 18 16:00:49.224503 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.224495    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc1d3388-4bec-44e2-b14e-b8bc5d70f465" containerName="istio-proxy"
Sep 18 16:00:49.224536 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.224504    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="785b2dc3-8249-46d9-9b0d-2d751e4c6434" containerName="test-curl-9134-98334b"
Sep 18 16:00:49.226753 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.226737    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-38e005"
Sep 18 16:00:49.235014 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.234990    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9134-38e005"]
Sep 18 16:00:49.242059 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:00:49.242038    2585 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9134-98334b" container="test-curl-9134-98334b"
Sep 18 16:00:49.365062 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.364952    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkgxt\" (UniqueName: \"kubernetes.io/projected/3bb77e9f-b829-4356-9f0e-75659a7fbb25-kube-api-access-kkgxt\") pod \"test-curl-9134-38e005\" (UID: \"3bb77e9f-b829-4356-9f0e-75659a7fbb25\") " pod="opendatahub/test-curl-9134-38e005"
Sep 18 16:00:49.466179 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.466143    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkgxt\" (UniqueName: \"kubernetes.io/projected/3bb77e9f-b829-4356-9f0e-75659a7fbb25-kube-api-access-kkgxt\") pod \"test-curl-9134-38e005\" (UID: \"3bb77e9f-b829-4356-9f0e-75659a7fbb25\") " pod="opendatahub/test-curl-9134-38e005"
Sep 18 16:00:49.474518 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.474491    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkgxt\" (UniqueName: \"kubernetes.io/projected/3bb77e9f-b829-4356-9f0e-75659a7fbb25-kube-api-access-kkgxt\") pod \"test-curl-9134-38e005\" (UID: \"3bb77e9f-b829-4356-9f0e-75659a7fbb25\") " pod="opendatahub/test-curl-9134-38e005"
Sep 18 16:00:49.537020 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.536980    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-38e005"
Sep 18 16:00:49.804796 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:49.804761    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9134-38e005"]
Sep 18 16:00:49.805771 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:00:49.805750    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3bb77e9f_b829_4356_9f0e_75659a7fbb25.slice/crio-6d67d7ad341294cd2156fc283f007aa628cace602078054fb6c05eb1d283f589 WatchSource:0}: Error finding container 6d67d7ad341294cd2156fc283f007aa628cace602078054fb6c05eb1d283f589: Status 404 returned error can't find the container with id 6d67d7ad341294cd2156fc283f007aa628cace602078054fb6c05eb1d283f589
Sep 18 16:00:50.250168 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:50.250136    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9134-38e005" event={"ID":"3bb77e9f-b829-4356-9f0e-75659a7fbb25","Type":"ContainerStarted","Data":"c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa"}
Sep 18 16:00:50.250168 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:50.250175    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9134-38e005" event={"ID":"3bb77e9f-b829-4356-9f0e-75659a7fbb25","Type":"ContainerStarted","Data":"6d67d7ad341294cd2156fc283f007aa628cace602078054fb6c05eb1d283f589"}
Sep 18 16:00:50.250640 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:50.250323    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9134-98334b" podUID="785b2dc3-8249-46d9-9b0d-2d751e4c6434" containerName="test-curl-9134-98334b" containerID="cri-o://28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a" gracePeriod=2
Sep 18 16:00:50.267897 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:50.267823    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9134-38e005" podStartSLOduration=1.267807909 podStartE2EDuration="1.267807909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:00:50.265861501 +0000 UTC m=+1574.924654153" watchObservedRunningTime="2026-09-18 16:00:50.267807909 +0000 UTC m=+1574.926600569"
Sep 18 16:00:50.933334 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:50.933299    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9134-38e005"]
Sep 18 16:00:50.936487 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:50.936454    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9134-38e005"]
Sep 18 16:00:51.255777 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:00:51.255747    2585 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9134-38e005" container="test-curl-9134-38e005"
Sep 18 16:00:52.261751 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:52.261719    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9134-38e005" podUID="3bb77e9f-b829-4356-9f0e-75659a7fbb25" containerName="test-curl-9134-38e005" containerID="cri-o://c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa" gracePeriod=2
Sep 18 16:00:52.397559 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:52.397522    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p"]
Sep 18 16:00:52.400831 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:52.400806    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-dc538f-29829120-lkk2p"]
Sep 18 16:00:52.453957 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:52.453924    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-98334b"
Sep 18 16:00:52.598348 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:52.598267    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sh9tn\" (UniqueName: \"kubernetes.io/projected/785b2dc3-8249-46d9-9b0d-2d751e4c6434-kube-api-access-sh9tn\") pod \"785b2dc3-8249-46d9-9b0d-2d751e4c6434\" (UID: \"785b2dc3-8249-46d9-9b0d-2d751e4c6434\") "
Sep 18 16:00:52.600274 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:52.600250    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785b2dc3-8249-46d9-9b0d-2d751e4c6434-kube-api-access-sh9tn" (OuterVolumeSpecName: "kube-api-access-sh9tn") pod "785b2dc3-8249-46d9-9b0d-2d751e4c6434" (UID: "785b2dc3-8249-46d9-9b0d-2d751e4c6434"). InnerVolumeSpecName "kube-api-access-sh9tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:00:52.699400 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:52.699361    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh9tn\" (UniqueName: \"kubernetes.io/projected/785b2dc3-8249-46d9-9b0d-2d751e4c6434-kube-api-access-sh9tn\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:53.269640 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:53.269599    2585 generic.go:356] "Generic (PLEG): container finished" podID="785b2dc3-8249-46d9-9b0d-2d751e4c6434" containerID="28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a" exitCode=137
Sep 18 16:00:53.270052 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:53.269647    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-98334b"
Sep 18 16:00:53.270052 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:53.269701    2585 scope.go:117] "RemoveContainer" containerID="28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a"
Sep 18 16:00:53.278829 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:53.278812    2585 scope.go:117] "RemoveContainer" containerID="28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a"
Sep 18 16:00:53.279094 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:00:53.279074    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a\": container with ID starting with 28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a not found: ID does not exist" containerID="28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a"
Sep 18 16:00:53.279145 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:53.279102    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a"} err="failed to get container status \"28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a\": rpc error: code = NotFound desc = could not find container \"28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a\": container with ID starting with 28ae0dc1e84624fec2268fffa724f2bd8b930b6a29f19eee5a3dafd24ca7e14a not found: ID does not exist"
Sep 18 16:00:53.280148 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:00:53.280124    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9134-98334b\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="785b2dc3-8249-46d9-9b0d-2d751e4c6434" pod="opendatahub/test-curl-9134-98334b"
Sep 18 16:00:53.523292 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:53.523208    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="785b2dc3-8249-46d9-9b0d-2d751e4c6434" path="/var/lib/kubelet/pods/785b2dc3-8249-46d9-9b0d-2d751e4c6434/volumes"
Sep 18 16:00:53.523500 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:53.523487    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dec6bdf9-4a8b-49ff-b703-8261497b87fa" path="/var/lib/kubelet/pods/dec6bdf9-4a8b-49ff-b703-8261497b87fa/volumes"
Sep 18 16:00:54.455687 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:54.455658    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-38e005"
Sep 18 16:00:54.620398 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:54.620304    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkgxt\" (UniqueName: \"kubernetes.io/projected/3bb77e9f-b829-4356-9f0e-75659a7fbb25-kube-api-access-kkgxt\") pod \"3bb77e9f-b829-4356-9f0e-75659a7fbb25\" (UID: \"3bb77e9f-b829-4356-9f0e-75659a7fbb25\") "
Sep 18 16:00:54.622318 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:54.622292    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bb77e9f-b829-4356-9f0e-75659a7fbb25-kube-api-access-kkgxt" (OuterVolumeSpecName: "kube-api-access-kkgxt") pod "3bb77e9f-b829-4356-9f0e-75659a7fbb25" (UID: "3bb77e9f-b829-4356-9f0e-75659a7fbb25"). InnerVolumeSpecName "kube-api-access-kkgxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:00:54.721454 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:54.721414    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkgxt\" (UniqueName: \"kubernetes.io/projected/3bb77e9f-b829-4356-9f0e-75659a7fbb25-kube-api-access-kkgxt\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:00:55.285519 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:55.285478    2585 generic.go:356] "Generic (PLEG): container finished" podID="3bb77e9f-b829-4356-9f0e-75659a7fbb25" containerID="c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa" exitCode=137
Sep 18 16:00:55.285519 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:55.285524    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9134-38e005"
Sep 18 16:00:55.285806 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:55.285585    2585 scope.go:117] "RemoveContainer" containerID="c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa"
Sep 18 16:00:55.296540 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:55.296517    2585 scope.go:117] "RemoveContainer" containerID="c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa"
Sep 18 16:00:55.296893 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:00:55.296866    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa\": container with ID starting with c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa not found: ID does not exist" containerID="c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa"
Sep 18 16:00:55.296989 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:55.296899    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa"} err="failed to get container status \"c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa\": rpc error: code = NotFound desc = could not find container \"c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa\": container with ID starting with c7d9961dda567ad1c9d560290e0566b18aaa78b208d2ef2a11fb3c31d32134fa not found: ID does not exist"
Sep 18 16:00:55.297993 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:00:55.297970    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9134-38e005\" is forbidden: User \"system:node:ip-10-0-130-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-208.ec2.internal' and this object" podUID="3bb77e9f-b829-4356-9f0e-75659a7fbb25" pod="opendatahub/test-curl-9134-38e005"
Sep 18 16:00:55.526544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:00:55.526503    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bb77e9f-b829-4356-9f0e-75659a7fbb25" path="/var/lib/kubelet/pods/3bb77e9f-b829-4356-9f0e-75659a7fbb25/volumes"
Sep 18 16:01:01.324564 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.324527    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"]
Sep 18 16:01:01.325242 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.324917    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bb77e9f-b829-4356-9f0e-75659a7fbb25" containerName="test-curl-9134-38e005"
Sep 18 16:01:01.325242 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.324929    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bb77e9f-b829-4356-9f0e-75659a7fbb25" containerName="test-curl-9134-38e005"
Sep 18 16:01:01.325242 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.325011    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bb77e9f-b829-4356-9f0e-75659a7fbb25" containerName="test-curl-9134-38e005"
Sep 18 16:01:01.327223 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.327203    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.331229 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.331204    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-f14aba\"/\"kube-root-ca.crt\""
Sep 18 16:01:01.331372 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.331246    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-f14aba\"/\"openshift-service-ca.crt\""
Sep 18 16:01:01.331372 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.331205    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-f14aba\"/\"rate-test-model-f14aba-kserve-self-signed-certs\""
Sep 18 16:01:01.331372 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.331203    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-f14aba\"/\"default-dockercfg-qpkcw\""
Sep 18 16:01:01.340636 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.340604    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"]
Sep 18 16:01:01.386303 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.386263    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-model-cache\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.386522 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.386326    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75afa9f1-9c83-4e25-b5a0-960510e6acce-tls-certs\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.386522 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.386428    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-dshm\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.386522 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.386485    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-tmp-dir\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.386522 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.386517    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9ftp\" (UniqueName: \"kubernetes.io/projected/75afa9f1-9c83-4e25-b5a0-960510e6acce-kube-api-access-h9ftp\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.386747 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.386598    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-home\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.487855 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.487804    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-dshm\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.488032 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.487877    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-tmp-dir\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.488032 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.487916    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9ftp\" (UniqueName: \"kubernetes.io/projected/75afa9f1-9c83-4e25-b5a0-960510e6acce-kube-api-access-h9ftp\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.488032 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.487956    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-home\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.488204 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.488068    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-model-cache\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.488204 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.488109    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75afa9f1-9c83-4e25-b5a0-960510e6acce-tls-certs\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.488328 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.488312    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-tmp-dir\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.488603 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.488577    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-model-cache\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.488763 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.488699    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-home\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.490289 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.490263    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-dshm\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.490551 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.490534    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75afa9f1-9c83-4e25-b5a0-960510e6acce-tls-certs\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.497082 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.497055    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9ftp\" (UniqueName: \"kubernetes.io/projected/75afa9f1-9c83-4e25-b5a0-960510e6acce-kube-api-access-h9ftp\") pod \"rate-test-model-f14aba-kserve-59fb8945c5-7drlh\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") " pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.636643 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.636557    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:01.916483 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:01.916401    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"]
Sep 18 16:01:01.916665 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:01:01.916634    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75afa9f1_9c83_4e25_b5a0_960510e6acce.slice/crio-f978dcc9bc5efbbb7d8c816ee89a2ddad83af71c2820c76e243c57c3b8bc1e37 WatchSource:0}: Error finding container f978dcc9bc5efbbb7d8c816ee89a2ddad83af71c2820c76e243c57c3b8bc1e37: Status 404 returned error can't find the container with id f978dcc9bc5efbbb7d8c816ee89a2ddad83af71c2820c76e243c57c3b8bc1e37
Sep 18 16:01:02.334946 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:02.334904    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh" event={"ID":"75afa9f1-9c83-4e25-b5a0-960510e6acce","Type":"ContainerStarted","Data":"10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec"}
Sep 18 16:01:02.335338 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:02.334955    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh" event={"ID":"75afa9f1-9c83-4e25-b5a0-960510e6acce","Type":"ContainerStarted","Data":"f978dcc9bc5efbbb7d8c816ee89a2ddad83af71c2820c76e243c57c3b8bc1e37"}
Sep 18 16:01:02.354324 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:02.354274    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh" podStartSLOduration=1.354260951 podStartE2EDuration="1.354260951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:01:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:01:02.35220929 +0000 UTC m=+1587.011001950" watchObservedRunningTime="2026-09-18 16:01:02.354260951 +0000 UTC m=+1587.013053611"
Sep 18 16:01:11.636900 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:11.636859    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:11.637407 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:11.636914    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:11.649589 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:11.649561    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:12.411176 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:12.411140    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:01:17.678034 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:17.677992    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"]
Sep 18 16:01:17.678524 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:17.678209    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy" containerID="cri-o://b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc" gracePeriod=30
Sep 18 16:01:22.934681 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:22.934654    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 16:01:23.096305 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096273    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-podinfo\") pod \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") "
Sep 18 16:01:23.096498 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096317    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-socket\") pod \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") "
Sep 18 16:01:23.096498 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096367    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpncc\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-kube-api-access-qpncc\") pod \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") "
Sep 18 16:01:23.096631 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096518    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istiod-ca-cert\") pod \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") "
Sep 18 16:01:23.096631 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096569    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-credential-socket\") pod \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") "
Sep 18 16:01:23.096631 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096592    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-data\") pod \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") "
Sep 18 16:01:23.096795 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096644    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24d01582-9f91-4fca-81e6-c0fe9a3dde29" (UID: "24d01582-9f91-4fca-81e6-c0fe9a3dde29"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:01:23.096795 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096666    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-envoy\") pod \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") "
Sep 18 16:01:23.096795 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096705    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-token\") pod \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") "
Sep 18 16:01:23.096795 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096764    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-certs\") pod \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\" (UID: \"24d01582-9f91-4fca-81e6-c0fe9a3dde29\") "
Sep 18 16:01:23.097028 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096901    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24d01582-9f91-4fca-81e6-c0fe9a3dde29" (UID: "24d01582-9f91-4fca-81e6-c0fe9a3dde29"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:01:23.097028 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096916    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-data" (OuterVolumeSpecName: "istio-data") pod "24d01582-9f91-4fca-81e6-c0fe9a3dde29" (UID: "24d01582-9f91-4fca-81e6-c0fe9a3dde29"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:01:23.097028 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.096918    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24d01582-9f91-4fca-81e6-c0fe9a3dde29" (UID: "24d01582-9f91-4fca-81e6-c0fe9a3dde29"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:01:23.097191 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.097137    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24d01582-9f91-4fca-81e6-c0fe9a3dde29" (UID: "24d01582-9f91-4fca-81e6-c0fe9a3dde29"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:01:23.097341 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.097322    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:23.097405 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.097342    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:23.097405 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.097351    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:23.097405 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.097360    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:23.097405 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.097369    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:23.098658 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.098629    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-kube-api-access-qpncc" (OuterVolumeSpecName: "kube-api-access-qpncc") pod "24d01582-9f91-4fca-81e6-c0fe9a3dde29" (UID: "24d01582-9f91-4fca-81e6-c0fe9a3dde29"). InnerVolumeSpecName "kube-api-access-qpncc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:01:23.098769 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.098687    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24d01582-9f91-4fca-81e6-c0fe9a3dde29" (UID: "24d01582-9f91-4fca-81e6-c0fe9a3dde29"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 16:01:23.098830 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.098806    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24d01582-9f91-4fca-81e6-c0fe9a3dde29" (UID: "24d01582-9f91-4fca-81e6-c0fe9a3dde29"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:01:23.099010 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.098992    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-token" (OuterVolumeSpecName: "istio-token") pod "24d01582-9f91-4fca-81e6-c0fe9a3dde29" (UID: "24d01582-9f91-4fca-81e6-c0fe9a3dde29"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:01:23.198417 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.198374    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:23.198417 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.198410    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:23.198417 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.198422    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24d01582-9f91-4fca-81e6-c0fe9a3dde29-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:23.198660 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.198434    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpncc\" (UniqueName: \"kubernetes.io/projected/24d01582-9f91-4fca-81e6-c0fe9a3dde29-kube-api-access-qpncc\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:23.463390 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.463292    2585 generic.go:356] "Generic (PLEG): container finished" podID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerID="b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc" exitCode=0
Sep 18 16:01:23.463390 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.463356    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"
Sep 18 16:01:23.463390 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.463371    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" event={"ID":"24d01582-9f91-4fca-81e6-c0fe9a3dde29","Type":"ContainerDied","Data":"b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc"}
Sep 18 16:01:23.463679 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.463474    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj" event={"ID":"24d01582-9f91-4fca-81e6-c0fe9a3dde29","Type":"ContainerDied","Data":"bff23c24a9ed467c64cb0024870315116b8e429400db8293f937a94f2b86664e"}
Sep 18 16:01:23.463679 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.463510    2585 scope.go:117] "RemoveContainer" containerID="b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc"
Sep 18 16:01:23.473137 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.473120    2585 scope.go:117] "RemoveContainer" containerID="325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73"
Sep 18 16:01:23.481127 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.481110    2585 scope.go:117] "RemoveContainer" containerID="b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc"
Sep 18 16:01:23.481386 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:01:23.481365    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc\": container with ID starting with b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc not found: ID does not exist" containerID="b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc"
Sep 18 16:01:23.481434 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.481395    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc"} err="failed to get container status \"b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc\": rpc error: code = NotFound desc = could not find container \"b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc\": container with ID starting with b456430212b7a62eb27a95be5af8372354711d616dd64128ca7db3ba258f22fc not found: ID does not exist"
Sep 18 16:01:23.481434 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.481410    2585 scope.go:117] "RemoveContainer" containerID="325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73"
Sep 18 16:01:23.481644 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:01:23.481629    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73\": container with ID starting with 325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73 not found: ID does not exist" containerID="325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73"
Sep 18 16:01:23.481693 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.481647    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73"} err="failed to get container status \"325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73\": rpc error: code = NotFound desc = could not find container \"325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73\": container with ID starting with 325dcc7b5f47c2d1f6acb8e9738e03438095219fd697136ea11368bf031a6b73 not found: ID does not exist"
Sep 18 16:01:23.486961 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.486932    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"]
Sep 18 16:01:23.492261 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.492240    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-195881-openshift-default-6c44bd8f76-zwzrj"]
Sep 18 16:01:23.524727 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:23.524702    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" path="/var/lib/kubelet/pods/24d01582-9f91-4fca-81e6-c0fe9a3dde29/volumes"
Sep 18 16:01:28.713360 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:28.713322    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"]
Sep 18 16:01:28.713888 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:28.713650    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm" podUID="abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" containerName="maas-api" containerID="cri-o://47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99" gracePeriod=30
Sep 18 16:01:28.959571 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:28.959544    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 16:01:29.057629 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.057599    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-metrics-certs\") pod \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") "
Sep 18 16:01:29.057833 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.057661    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qj45\" (UniqueName: \"kubernetes.io/projected/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-kube-api-access-4qj45\") pod \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") "
Sep 18 16:01:29.057833 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.057778    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-openshift-service-ca\") pod \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") "
Sep 18 16:01:29.057833 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.057806    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-maas-api-tls\") pod \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\" (UID: \"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1\") "
Sep 18 16:01:29.058148 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.058123    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" (UID: "abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:01:29.059921 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.059896    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-kube-api-access-4qj45" (OuterVolumeSpecName: "kube-api-access-4qj45") pod "abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" (UID: "abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1"). InnerVolumeSpecName "kube-api-access-4qj45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:01:29.059921 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.059915    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" (UID: "abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:01:29.060047 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.059970    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" (UID: "abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:01:29.159002 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.158964    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:29.159002 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.158995    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:29.159002 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.159008    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:29.159230 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.159020    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qj45\" (UniqueName: \"kubernetes.io/projected/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1-kube-api-access-4qj45\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:01:29.504577 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.504546    2585 generic.go:356] "Generic (PLEG): container finished" podID="abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" containerID="47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99" exitCode=0
Sep 18 16:01:29.504748 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.504596    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"
Sep 18 16:01:29.504748 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.504627    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm" event={"ID":"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1","Type":"ContainerDied","Data":"47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99"}
Sep 18 16:01:29.504748 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.504679    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm" event={"ID":"abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1","Type":"ContainerDied","Data":"d399c33c3a36716d5141ea62c57785711abdddaefe25a61402e85f83c59efbca"}
Sep 18 16:01:29.504748 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.504701    2585 scope.go:117] "RemoveContainer" containerID="47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99"
Sep 18 16:01:29.515156 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.515140    2585 scope.go:117] "RemoveContainer" containerID="47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99"
Sep 18 16:01:29.515406 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:01:29.515386    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99\": container with ID starting with 47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99 not found: ID does not exist" containerID="47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99"
Sep 18 16:01:29.515513 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.515418    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99"} err="failed to get container status \"47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99\": rpc error: code = NotFound desc = could not find container \"47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99\": container with ID starting with 47aa0753fa434c681366c6c9afb9d6ea34a122c030a5d5c01ef3359647758e99 not found: ID does not exist"
Sep 18 16:01:29.529335 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.529310    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"]
Sep 18 16:01:29.532353 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:29.532334    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-655d9c-7d95cb9549-6hvxm"]
Sep 18 16:01:31.524226 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:01:31.524191    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" path="/var/lib/kubelet/pods/abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1/volumes"
Sep 18 16:02:08.794343 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:08.794309    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"]
Sep 18 16:02:10.626901 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.626856    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"]
Sep 18 16:02:10.627305 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.627269    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" containerName="maas-api"
Sep 18 16:02:10.627305 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.627281    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" containerName="maas-api"
Sep 18 16:02:10.627305 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.627296    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy"
Sep 18 16:02:10.627305 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.627302    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy"
Sep 18 16:02:10.627544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.627317    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy"
Sep 18 16:02:10.627544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.627323    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy"
Sep 18 16:02:10.627544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.627376    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy"
Sep 18 16:02:10.627544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.627385    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="24d01582-9f91-4fca-81e6-c0fe9a3dde29" containerName="istio-proxy"
Sep 18 16:02:10.627544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.627392    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf0ed09-ead9-43b2-a4d6-4103ca2f6bc1" containerName="maas-api"
Sep 18 16:02:10.629492 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.629472    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.632502 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.632481    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-90ebe5-openshift-default-dockercfg-gvddg\""
Sep 18 16:02:10.643823 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.643802    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"]
Sep 18 16:02:10.729555 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.729515    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c09bec6-33e1-4349-9907-e92523ad1367-istiod-ca-cert\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.729733 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.729640    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-istio-token\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.729733 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.729687    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-socket\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.729846 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.729775    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c09bec6-33e1-4349-9907-e92523ad1367-istio-podinfo\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.729846 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.729799    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfl4g\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-kube-api-access-xfl4g\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.729846 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.729826    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-envoy\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.729988 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.729873    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-data\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.729988 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.729928    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-credential-socket\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.729988 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.729950    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-certs\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.831454 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.831422    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-istio-token\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.831624 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.831479    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-socket\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.831700 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.831660    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c09bec6-33e1-4349-9907-e92523ad1367-istio-podinfo\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.831758 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.831698    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfl4g\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-kube-api-access-xfl4g\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.831758 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.831728    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-envoy\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.831879 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.831831    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-data\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.831945 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.831892    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-socket\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.831945 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.831909    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-credential-socket\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.831945 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.831939    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-certs\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.832328 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.832196    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-data\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.832328 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.832206    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c09bec6-33e1-4349-9907-e92523ad1367-istiod-ca-cert\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.832328 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.832262    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-credential-socket\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.832328 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.832278    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-certs\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.832974 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.832950    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c09bec6-33e1-4349-9907-e92523ad1367-istiod-ca-cert\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.833970 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.833952    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-envoy\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.834395 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.834363    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c09bec6-33e1-4349-9907-e92523ad1367-istio-podinfo\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.840534 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.840514    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-istio-token\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.841159 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.841112    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfl4g\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-kube-api-access-xfl4g\") pod \"e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") " pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:10.940124 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:10.940046    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:11.220515 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:11.220477    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"]
Sep 18 16:02:11.222055 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:02:11.222032    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c09bec6_33e1_4349_9907_e92523ad1367.slice/crio-1dbeefdc31086dca5d4fbff475918814b63d6fb74f1b154088efe602e64bf176 WatchSource:0}: Error finding container 1dbeefdc31086dca5d4fbff475918814b63d6fb74f1b154088efe602e64bf176: Status 404 returned error can't find the container with id 1dbeefdc31086dca5d4fbff475918814b63d6fb74f1b154088efe602e64bf176
Sep 18 16:02:11.224303 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:11.224273    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:02:11.224401 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:11.224349    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:02:11.224466 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:11.224396    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:02:11.758435 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:11.758402    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" event={"ID":"4c09bec6-33e1-4349-9907-e92523ad1367","Type":"ContainerStarted","Data":"76499f0d565f44c3bc94fd4f2e3c9c808969c86615a3321e727344bc8dced362"}
Sep 18 16:02:11.758853 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:11.758452    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" event={"ID":"4c09bec6-33e1-4349-9907-e92523ad1367","Type":"ContainerStarted","Data":"1dbeefdc31086dca5d4fbff475918814b63d6fb74f1b154088efe602e64bf176"}
Sep 18 16:02:11.785454 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:11.785397    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" podStartSLOduration=1.785383151 podStartE2EDuration="1.785383151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:02:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:02:11.782235904 +0000 UTC m=+1656.441028579" watchObservedRunningTime="2026-09-18 16:02:11.785383151 +0000 UTC m=+1656.444175811"
Sep 18 16:02:11.940322 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:11.940295    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:12.946066 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:12.946036    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:13.769755 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:13.769722    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:13.770700 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:13.770679    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:23.816449 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:23.816400    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh" podUID="75afa9f1-9c83-4e25-b5a0-960510e6acce" containerName="main" containerID="cri-o://10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec" gracePeriod=45
Sep 18 16:02:24.091488 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.091461    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:02:24.166378 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.166339    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75afa9f1-9c83-4e25-b5a0-960510e6acce-tls-certs\") pod \"75afa9f1-9c83-4e25-b5a0-960510e6acce\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") "
Sep 18 16:02:24.166562 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.166416    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-model-cache\") pod \"75afa9f1-9c83-4e25-b5a0-960510e6acce\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") "
Sep 18 16:02:24.166919 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.166443    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-tmp-dir\") pod \"75afa9f1-9c83-4e25-b5a0-960510e6acce\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") "
Sep 18 16:02:24.167077 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.166944    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "75afa9f1-9c83-4e25-b5a0-960510e6acce" (UID: "75afa9f1-9c83-4e25-b5a0-960510e6acce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:02:24.167077 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.167032    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-home\") pod \"75afa9f1-9c83-4e25-b5a0-960510e6acce\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") "
Sep 18 16:02:24.167077 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.167043    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-model-cache" (OuterVolumeSpecName: "model-cache") pod "75afa9f1-9c83-4e25-b5a0-960510e6acce" (UID: "75afa9f1-9c83-4e25-b5a0-960510e6acce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:02:24.167077 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.167076    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-dshm\") pod \"75afa9f1-9c83-4e25-b5a0-960510e6acce\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") "
Sep 18 16:02:24.167303 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.167235    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-home" (OuterVolumeSpecName: "home") pod "75afa9f1-9c83-4e25-b5a0-960510e6acce" (UID: "75afa9f1-9c83-4e25-b5a0-960510e6acce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:02:24.167303 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.167274    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9ftp\" (UniqueName: \"kubernetes.io/projected/75afa9f1-9c83-4e25-b5a0-960510e6acce-kube-api-access-h9ftp\") pod \"75afa9f1-9c83-4e25-b5a0-960510e6acce\" (UID: \"75afa9f1-9c83-4e25-b5a0-960510e6acce\") "
Sep 18 16:02:24.167821 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.167796    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-home\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:02:24.167956 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.167825    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-model-cache\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:02:24.167956 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.167868    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-tmp-dir\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:02:24.173880 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.171491    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-dshm" (OuterVolumeSpecName: "dshm") pod "75afa9f1-9c83-4e25-b5a0-960510e6acce" (UID: "75afa9f1-9c83-4e25-b5a0-960510e6acce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:02:24.175273 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.175245    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75afa9f1-9c83-4e25-b5a0-960510e6acce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "75afa9f1-9c83-4e25-b5a0-960510e6acce" (UID: "75afa9f1-9c83-4e25-b5a0-960510e6acce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:02:24.175543 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.175518    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75afa9f1-9c83-4e25-b5a0-960510e6acce-kube-api-access-h9ftp" (OuterVolumeSpecName: "kube-api-access-h9ftp") pod "75afa9f1-9c83-4e25-b5a0-960510e6acce" (UID: "75afa9f1-9c83-4e25-b5a0-960510e6acce"). InnerVolumeSpecName "kube-api-access-h9ftp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:02:24.269253 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.269217    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75afa9f1-9c83-4e25-b5a0-960510e6acce-dshm\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:02:24.269253 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.269245    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9ftp\" (UniqueName: \"kubernetes.io/projected/75afa9f1-9c83-4e25-b5a0-960510e6acce-kube-api-access-h9ftp\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:02:24.269253 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.269260    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75afa9f1-9c83-4e25-b5a0-960510e6acce-tls-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:02:24.840434 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.840398    2585 generic.go:356] "Generic (PLEG): container finished" podID="75afa9f1-9c83-4e25-b5a0-960510e6acce" containerID="10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec" exitCode=0
Sep 18 16:02:24.840860 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.840458    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"
Sep 18 16:02:24.840860 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.840480    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh" event={"ID":"75afa9f1-9c83-4e25-b5a0-960510e6acce","Type":"ContainerDied","Data":"10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec"}
Sep 18 16:02:24.840860 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.840524    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh" event={"ID":"75afa9f1-9c83-4e25-b5a0-960510e6acce","Type":"ContainerDied","Data":"f978dcc9bc5efbbb7d8c816ee89a2ddad83af71c2820c76e243c57c3b8bc1e37"}
Sep 18 16:02:24.840860 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.840541    2585 scope.go:117] "RemoveContainer" containerID="10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec"
Sep 18 16:02:24.850200 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.850185    2585 scope.go:117] "RemoveContainer" containerID="10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec"
Sep 18 16:02:24.850455 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:02:24.850437    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec\": container with ID starting with 10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec not found: ID does not exist" containerID="10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec"
Sep 18 16:02:24.850508 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.850464    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec"} err="failed to get container status \"10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec\": rpc error: code = NotFound desc = could not find container \"10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec\": container with ID starting with 10956155fa0212ed79224638baab874a95feb589c857083bdba5980394a89eec not found: ID does not exist"
Sep 18 16:02:24.872030 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.872005    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"]
Sep 18 16:02:24.885229 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:24.885200    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f14aba/rate-test-model-f14aba-kserve-59fb8945c5-7drlh"]
Sep 18 16:02:25.527725 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:25.527676    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75afa9f1-9c83-4e25-b5a0-960510e6acce" path="/var/lib/kubelet/pods/75afa9f1-9c83-4e25-b5a0-960510e6acce/volumes"
Sep 18 16:02:29.329231 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:29.329190    2585 patch_prober.go:28] interesting pod/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": read tcp 10.133.0.2:56580->10.133.0.73:15021: read: connection reset by peer" start-of-body=
Sep 18 16:02:29.329701 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:29.329264    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": read tcp 10.133.0.2:56580->10.133.0.73:15021: read: connection reset by peer"
Sep 18 16:02:29.872881 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:29.872825    2585 generic.go:356] "Generic (PLEG): container finished" podID="4c09bec6-33e1-4349-9907-e92523ad1367" containerID="76499f0d565f44c3bc94fd4f2e3c9c808969c86615a3321e727344bc8dced362" exitCode=0
Sep 18 16:02:29.873055 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:29.872896    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" event={"ID":"4c09bec6-33e1-4349-9907-e92523ad1367","Type":"ContainerDied","Data":"76499f0d565f44c3bc94fd4f2e3c9c808969c86615a3321e727344bc8dced362"}
Sep 18 16:02:29.873267 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:29.873253    2585 scope.go:117] "RemoveContainer" containerID="76499f0d565f44c3bc94fd4f2e3c9c808969c86615a3321e727344bc8dced362"
Sep 18 16:02:29.874271 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:29.874246    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:02:29.874335 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:29.874315    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:02:29.874373 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:29.874354    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:02:30.880569 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:30.880531    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" event={"ID":"4c09bec6-33e1-4349-9907-e92523ad1367","Type":"ContainerStarted","Data":"4d6d60def15db9bb515b1045d902fd827e3afe15ae511377cd22e3ca87d01abd"}
Sep 18 16:02:30.940330 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:30.940294    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:30.940509 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:30.940453    2585 patch_prober.go:28] interesting pod/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 18 16:02:30.940565 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:30.940505    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 18 16:02:31.940681 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:31.940638    2585 patch_prober.go:28] interesting pod/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 18 16:02:31.941197 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:31.940705    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 18 16:02:32.941349 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:32.941312    2585 patch_prober.go:28] interesting pod/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 18 16:02:32.941770 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:32.941369    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 18 16:02:33.940386 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:33.940347    2585 patch_prober.go:28] interesting pod/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 18 16:02:33.940566 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:33.940408    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 18 16:02:34.940453 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:34.940413    2585 patch_prober.go:28] interesting pod/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 18 16:02:34.940952 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:34.940477    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 18 16:02:35.940700 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:35.940652    2585 patch_prober.go:28] interesting pod/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 18 16:02:35.941160 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:35.940729    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 18 16:02:36.944470 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:36.944442    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:36.944866 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:36.944723    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:36.945365 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:36.945346    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:02:45.254455 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.254422    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"]
Sep 18 16:02:45.254930 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.254849    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75afa9f1-9c83-4e25-b5a0-960510e6acce" containerName="main"
Sep 18 16:02:45.254930 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.254864    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="75afa9f1-9c83-4e25-b5a0-960510e6acce" containerName="main"
Sep 18 16:02:45.254930 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.254930    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="75afa9f1-9c83-4e25-b5a0-960510e6acce" containerName="main"
Sep 18 16:02:45.259513 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.259491    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.264893 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.264871    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-e41875-openshift-default-dockercfg-f9bzp\""
Sep 18 16:02:45.276951 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.276930    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"]
Sep 18 16:02:45.369655 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.369046    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-credential-socket\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.369655 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.369100    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-certs\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.369655 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.369148    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-socket\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.369655 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.369176    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-data\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.369655 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.369254    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6498153f-ce82-4c89-971f-ee1f2d66bc94-istiod-ca-cert\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.369655 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.369299    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-envoy\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.369655 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.369388    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-podinfo\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.369655 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.369434    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2h5x\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-kube-api-access-z2h5x\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.369655 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.369462    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-token\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.470536 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.470509    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-podinfo\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.470720 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.470545    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2h5x\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-kube-api-access-z2h5x\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.470720 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.470566    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-token\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.470720 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.470587    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-credential-socket\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.470720 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.470605    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-certs\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.470720 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.470641    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-socket\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.470720 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.470667    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-data\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.471107 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.470730    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6498153f-ce82-4c89-971f-ee1f2d66bc94-istiod-ca-cert\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.471107 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.470771    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-envoy\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.471107 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.471049    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-credential-socket\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.471234 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.471101    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-data\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.471234 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.471136    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-certs\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.471322 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.471278    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-socket\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.471771 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.471748    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6498153f-ce82-4c89-971f-ee1f2d66bc94-istiod-ca-cert\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.473047 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.473030    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-envoy\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.473422 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.473407    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-podinfo\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.484794 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.484767    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-token\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.488442 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.488421    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2h5x\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-kube-api-access-z2h5x\") pod \"e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") " pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.570253 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.570175    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:45.859471 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.859444    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"]
Sep 18 16:02:45.860928 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:02:45.860902    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6498153f_ce82_4c89_971f_ee1f2d66bc94.slice/crio-2d3bbfaf0e19beda169cbcd0e33e49925d0744937f48c831d5ad136c12b8b66e WatchSource:0}: Error finding container 2d3bbfaf0e19beda169cbcd0e33e49925d0744937f48c831d5ad136c12b8b66e: Status 404 returned error can't find the container with id 2d3bbfaf0e19beda169cbcd0e33e49925d0744937f48c831d5ad136c12b8b66e
Sep 18 16:02:45.863190 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.863158    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:02:45.863263 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.863223    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:02:45.863263 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.863256    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:02:45.975253 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.975215    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" event={"ID":"6498153f-ce82-4c89-971f-ee1f2d66bc94","Type":"ContainerStarted","Data":"0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07"}
Sep 18 16:02:45.975457 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.975304    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" event={"ID":"6498153f-ce82-4c89-971f-ee1f2d66bc94","Type":"ContainerStarted","Data":"2d3bbfaf0e19beda169cbcd0e33e49925d0744937f48c831d5ad136c12b8b66e"}
Sep 18 16:02:45.998775 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:45.998721    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" podStartSLOduration=0.998702838 podStartE2EDuration="998.702838ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:02:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:02:45.996885899 +0000 UTC m=+1690.655678560" watchObservedRunningTime="2026-09-18 16:02:45.998702838 +0000 UTC m=+1690.657495497"
Sep 18 16:02:46.571177 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:46.571142    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:46.576296 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:46.576273    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:46.980441 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:46.980366    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:02:46.981132 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:02:46.981111    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:03:03.023029 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:03.022984    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 16:03:03.023428 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:03.023060    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 16:03:05.094623 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:05.094579    2585 generic.go:356] "Generic (PLEG): container finished" podID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerID="0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07" exitCode=0
Sep 18 16:03:05.095197 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:05.094654    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" event={"ID":"6498153f-ce82-4c89-971f-ee1f2d66bc94","Type":"ContainerDied","Data":"0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07"}
Sep 18 16:03:05.095197 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:05.095175    2585 scope.go:117] "RemoveContainer" containerID="0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07"
Sep 18 16:03:05.096382 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:05.096353    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:03:05.096443 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:05.096431    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:03:05.096479 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:05.096464    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:03:06.102344 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:06.102314    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" event={"ID":"6498153f-ce82-4c89-971f-ee1f2d66bc94","Type":"ContainerStarted","Data":"cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190"}
Sep 18 16:03:06.570848 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:06.570816    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:03:06.571068 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:06.571047    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 18 16:03:06.571151 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:06.571132    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 18 16:03:07.571068 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:07.571032    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 18 16:03:07.571449 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:07.571086    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 18 16:03:08.570960 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:08.570920    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 18 16:03:08.571178 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:08.570987    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 18 16:03:09.571199 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:09.571158    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 18 16:03:09.571670 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:09.571218    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 18 16:03:10.571584 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:10.571547    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 18 16:03:10.572018 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:10.571605    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 18 16:03:11.574189 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:11.574158    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:03:11.574725 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:11.574433    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:03:11.575126 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:03:11.575106    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:04:10.932417 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:10.932384    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"]
Sep 18 16:04:10.935236 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:10.935213    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:10.938003 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:10.937986    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-e2585e-openshift-default-dockercfg-hnx8n\""
Sep 18 16:04:10.948783 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:10.948758    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"]
Sep 18 16:04:11.073907 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.073869    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-envoy\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.074099 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.073944    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-credential-socket\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.074099 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.073972    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-data\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.074099 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.074003    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-podinfo\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.074099 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.074036    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzs2h\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-kube-api-access-fzs2h\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.074327 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.074121    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-token\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.074327 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.074167    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-certs\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.074327 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.074229    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-socket\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.074327 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.074275    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.175422 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.175384    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-socket\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.175593 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.175432    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.175593 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.175549    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-envoy\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.175685 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.175624    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-credential-socket\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.175685 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.175647    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-data\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.175791 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.175690    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-podinfo\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.175791 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.175733    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzs2h\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-kube-api-access-fzs2h\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.175911 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.175855    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-socket\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.176051 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.176023    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-credential-socket\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.176186 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.176128    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-data\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.176251 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.176217    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-token\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.176316 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.176255    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-certs\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.176316 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.176260    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.176639 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.176615    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-certs\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.177992 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.177974    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-envoy\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.178276 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.178258    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-podinfo\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.185514 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.185450    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzs2h\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-kube-api-access-fzs2h\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.185514 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.185457    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-token\") pod \"e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") " pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.246039 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.246008    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:11.526342 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.526316    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"]
Sep 18 16:04:11.527025 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:04:11.527000    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c7b919_7ac4_483c_a4a2_d21f5e9f3385.slice/crio-702e404515f7d56992f28d07dee217bdd9e60d61282d404e5b521a66c8711ded WatchSource:0}: Error finding container 702e404515f7d56992f28d07dee217bdd9e60d61282d404e5b521a66c8711ded: Status 404 returned error can't find the container with id 702e404515f7d56992f28d07dee217bdd9e60d61282d404e5b521a66c8711ded
Sep 18 16:04:11.529458 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.529420    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:04:11.529539 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.529490    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:04:11.529539 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:11.529514    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:04:12.486355 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:12.486318    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" event={"ID":"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385","Type":"ContainerStarted","Data":"9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e"}
Sep 18 16:04:12.486774 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:12.486373    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" event={"ID":"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385","Type":"ContainerStarted","Data":"702e404515f7d56992f28d07dee217bdd9e60d61282d404e5b521a66c8711ded"}
Sep 18 16:04:12.514416 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:12.514360    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" podStartSLOduration=2.514344634 podStartE2EDuration="2.514344634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:04:12.51312694 +0000 UTC m=+1777.171919596" watchObservedRunningTime="2026-09-18 16:04:12.514344634 +0000 UTC m=+1777.173137293"
Sep 18 16:04:13.246539 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:13.246498    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:13.252045 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:13.252022    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:13.493568 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:13.493534    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:13.494559 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:13.494538    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:33.616620 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:33.616585    2585 generic.go:356] "Generic (PLEG): container finished" podID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerID="9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e" exitCode=0
Sep 18 16:04:33.617058 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:33.616660    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" event={"ID":"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385","Type":"ContainerDied","Data":"9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e"}
Sep 18 16:04:33.617117 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:33.617101    2585 scope.go:117] "RemoveContainer" containerID="9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e"
Sep 18 16:04:33.618447 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:33.618417    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:04:33.618523 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:33.618507    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:04:33.618569 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:33.618554    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 16:04:34.629898 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:34.629857    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" event={"ID":"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385","Type":"ContainerStarted","Data":"fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3"}
Sep 18 16:04:35.246933 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:35.246884    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:35.247146 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:35.247121    2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 16:04:35.247194 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:35.247176    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 16:04:36.247271 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:36.247236    2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 16:04:36.247653 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:36.247303    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 16:04:37.246797 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:37.246762    2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 16:04:37.247003 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:37.246820    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 16:04:37.604705 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:37.604633    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 16:04:37.636094 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:37.635959    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 16:04:38.247267 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:38.247230    2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 16:04:38.247457 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:38.247301    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 16:04:39.247115 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:39.247075    2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 16:04:39.247523 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:39.247148    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 16:04:40.250921 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:40.250883    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:40.251403 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:40.251302    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:04:40.252047 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:04:40.252025    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:05:03.455968 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:03.455932    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"]
Sep 18 16:05:03.456493 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:03.456152    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy" containerID="cri-o://fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3" gracePeriod=30
Sep 18 16:05:08.705224 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.705199    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:05:08.757892 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.757830    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-certs\") pod \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") "
Sep 18 16:05:08.758062 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.757982    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-podinfo\") pod \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") "
Sep 18 16:05:08.758062 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758028    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-data\") pod \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") "
Sep 18 16:05:08.758191 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758061    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzs2h\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-kube-api-access-fzs2h\") pod \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") "
Sep 18 16:05:08.758191 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758081    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" (UID: "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:08.758191 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758128    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-token\") pod \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") "
Sep 18 16:05:08.758191 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758159    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istiod-ca-cert\") pod \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") "
Sep 18 16:05:08.758400 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758217    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-credential-socket\") pod \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") "
Sep 18 16:05:08.758400 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758251    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-data" (OuterVolumeSpecName: "istio-data") pod "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" (UID: "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:08.758400 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758298    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-envoy\") pod \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") "
Sep 18 16:05:08.758400 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758326    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-socket\") pod \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\" (UID: \"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385\") "
Sep 18 16:05:08.758619 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758468    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" (UID: "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:08.758619 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758554    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" (UID: "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:05:08.758729 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758632    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" (UID: "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:08.758812 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758791    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:08.758883 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758819    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:08.758883 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758834    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:08.758883 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758864    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:08.758883 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.758877    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:08.760468 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.760439    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-token" (OuterVolumeSpecName: "istio-token") pod "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" (UID: "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:05:08.760468 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.760460    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-kube-api-access-fzs2h" (OuterVolumeSpecName: "kube-api-access-fzs2h") pod "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" (UID: "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385"). InnerVolumeSpecName "kube-api-access-fzs2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:05:08.760651 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.760460    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" (UID: "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 16:05:08.760824 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.760804    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" (UID: "c0c7b919-7ac4-483c-a4a2-d21f5e9f3385"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:08.834349 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.834316    2585 generic.go:356] "Generic (PLEG): container finished" podID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerID="fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3" exitCode=0
Sep 18 16:05:08.834544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.834370    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"
Sep 18 16:05:08.834544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.834400    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" event={"ID":"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385","Type":"ContainerDied","Data":"fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3"}
Sep 18 16:05:08.834544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.834458    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj" event={"ID":"c0c7b919-7ac4-483c-a4a2-d21f5e9f3385","Type":"ContainerDied","Data":"702e404515f7d56992f28d07dee217bdd9e60d61282d404e5b521a66c8711ded"}
Sep 18 16:05:08.834544 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.834490    2585 scope.go:117] "RemoveContainer" containerID="fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3"
Sep 18 16:05:08.844595 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.844574    2585 scope.go:117] "RemoveContainer" containerID="9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e"
Sep 18 16:05:08.852988 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.852972    2585 scope.go:117] "RemoveContainer" containerID="fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3"
Sep 18 16:05:08.853236 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:05:08.853217    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3\": container with ID starting with fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3 not found: ID does not exist" containerID="fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3"
Sep 18 16:05:08.853316 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.853249    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3"} err="failed to get container status \"fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3\": rpc error: code = NotFound desc = could not find container \"fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3\": container with ID starting with fe768615a17b670db7e413445e5c3dd65a309b70d39b51f1855e5a76825d1ff3 not found: ID does not exist"
Sep 18 16:05:08.853316 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.853272    2585 scope.go:117] "RemoveContainer" containerID="9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e"
Sep 18 16:05:08.853533 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:05:08.853516    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e\": container with ID starting with 9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e not found: ID does not exist" containerID="9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e"
Sep 18 16:05:08.853573 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.853541    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e"} err="failed to get container status \"9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e\": rpc error: code = NotFound desc = could not find container \"9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e\": container with ID starting with 9ac8a9e23dce2208991317db121a6bcfb5dc0aae5603ecae5129f117aecca33e not found: ID does not exist"
Sep 18 16:05:08.859426 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.859404    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:08.859535 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.859431    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:08.859535 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.859445    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzs2h\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-kube-api-access-fzs2h\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:08.859535 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.859459    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:08.860024 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.860011    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"]
Sep 18 16:05:08.862380 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:08.862359    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e2585e-openshift-default-56f89b75c6-nvntj"]
Sep 18 16:05:09.525124 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:09.525091    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" path="/var/lib/kubelet/pods/c0c7b919-7ac4-483c-a4a2-d21f5e9f3385/volumes"
Sep 18 16:05:52.978658 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:52.978567    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"]
Sep 18 16:05:52.979241 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:52.978864    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy" containerID="cri-o://4d6d60def15db9bb515b1045d902fd827e3afe15ae511377cd22e3ca87d01abd" gracePeriod=30
Sep 18 16:05:58.129333 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.129296    2585 generic.go:356] "Generic (PLEG): container finished" podID="4c09bec6-33e1-4349-9907-e92523ad1367" containerID="4d6d60def15db9bb515b1045d902fd827e3afe15ae511377cd22e3ca87d01abd" exitCode=0
Sep 18 16:05:58.129790 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.129377    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" event={"ID":"4c09bec6-33e1-4349-9907-e92523ad1367","Type":"ContainerDied","Data":"4d6d60def15db9bb515b1045d902fd827e3afe15ae511377cd22e3ca87d01abd"}
Sep 18 16:05:58.129790 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.129456    2585 scope.go:117] "RemoveContainer" containerID="76499f0d565f44c3bc94fd4f2e3c9c808969c86615a3321e727344bc8dced362"
Sep 18 16:05:58.231735 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.231710    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:05:58.338774 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.338688    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfl4g\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-kube-api-access-xfl4g\") pod \"4c09bec6-33e1-4349-9907-e92523ad1367\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") "
Sep 18 16:05:58.338774 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.338739    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c09bec6-33e1-4349-9907-e92523ad1367-istiod-ca-cert\") pod \"4c09bec6-33e1-4349-9907-e92523ad1367\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") "
Sep 18 16:05:58.338774 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.338775    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-credential-socket\") pod \"4c09bec6-33e1-4349-9907-e92523ad1367\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") "
Sep 18 16:05:58.339096 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.338793    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-certs\") pod \"4c09bec6-33e1-4349-9907-e92523ad1367\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") "
Sep 18 16:05:58.339096 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.338818    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-data\") pod \"4c09bec6-33e1-4349-9907-e92523ad1367\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") "
Sep 18 16:05:58.339096 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.338877    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-envoy\") pod \"4c09bec6-33e1-4349-9907-e92523ad1367\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") "
Sep 18 16:05:58.339096 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.338942    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-istio-token\") pod \"4c09bec6-33e1-4349-9907-e92523ad1367\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") "
Sep 18 16:05:58.339096 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.338962    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-socket\") pod \"4c09bec6-33e1-4349-9907-e92523ad1367\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") "
Sep 18 16:05:58.339096 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.338993    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c09bec6-33e1-4349-9907-e92523ad1367-istio-podinfo\") pod \"4c09bec6-33e1-4349-9907-e92523ad1367\" (UID: \"4c09bec6-33e1-4349-9907-e92523ad1367\") "
Sep 18 16:05:58.339411 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.339173    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c09bec6-33e1-4349-9907-e92523ad1367" (UID: "4c09bec6-33e1-4349-9907-e92523ad1367"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:58.339411 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.339189    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c09bec6-33e1-4349-9907-e92523ad1367" (UID: "4c09bec6-33e1-4349-9907-e92523ad1367"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:58.339411 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.339201    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c09bec6-33e1-4349-9907-e92523ad1367" (UID: "4c09bec6-33e1-4349-9907-e92523ad1367"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:58.339411 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.339267    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c09bec6-33e1-4349-9907-e92523ad1367-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c09bec6-33e1-4349-9907-e92523ad1367" (UID: "4c09bec6-33e1-4349-9907-e92523ad1367"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:05:58.339411 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.339394    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:58.339681 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.339416    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:58.339681 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.339420    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c09bec6-33e1-4349-9907-e92523ad1367" (UID: "4c09bec6-33e1-4349-9907-e92523ad1367"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:58.339681 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.339431    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:58.339681 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.339466    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c09bec6-33e1-4349-9907-e92523ad1367-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:58.341244 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.341216    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c09bec6-33e1-4349-9907-e92523ad1367" (UID: "4c09bec6-33e1-4349-9907-e92523ad1367"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:05:58.341353 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.341251    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c09bec6-33e1-4349-9907-e92523ad1367" (UID: "4c09bec6-33e1-4349-9907-e92523ad1367"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:05:58.341353 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.341341    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c09bec6-33e1-4349-9907-e92523ad1367-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c09bec6-33e1-4349-9907-e92523ad1367" (UID: "4c09bec6-33e1-4349-9907-e92523ad1367"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 16:05:58.341446 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.341341    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-kube-api-access-xfl4g" (OuterVolumeSpecName: "kube-api-access-xfl4g") pod "4c09bec6-33e1-4349-9907-e92523ad1367" (UID: "4c09bec6-33e1-4349-9907-e92523ad1367"). InnerVolumeSpecName "kube-api-access-xfl4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:05:58.440261 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.440221    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:58.440261 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.440255    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:58.440261 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.440264    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c09bec6-33e1-4349-9907-e92523ad1367-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:58.440261 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.440272    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfl4g\" (UniqueName: \"kubernetes.io/projected/4c09bec6-33e1-4349-9907-e92523ad1367-kube-api-access-xfl4g\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:58.440538 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:58.440281    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c09bec6-33e1-4349-9907-e92523ad1367-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:05:59.136338 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:59.136311    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"
Sep 18 16:05:59.136774 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:59.136353    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf" event={"ID":"4c09bec6-33e1-4349-9907-e92523ad1367","Type":"ContainerDied","Data":"1dbeefdc31086dca5d4fbff475918814b63d6fb74f1b154088efe602e64bf176"}
Sep 18 16:05:59.136774 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:59.136414    2585 scope.go:117] "RemoveContainer" containerID="4d6d60def15db9bb515b1045d902fd827e3afe15ae511377cd22e3ca87d01abd"
Sep 18 16:05:59.159106 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:59.159082    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"]
Sep 18 16:05:59.163574 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:59.163553    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-90ebe5-openshift-default-897b7bd98-2qtpf"]
Sep 18 16:05:59.523834 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:05:59.523803    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" path="/var/lib/kubelet/pods/4c09bec6-33e1-4349-9907-e92523ad1367/volumes"
Sep 18 16:06:11.166640 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:11.166608    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"]
Sep 18 16:06:11.167130 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:11.166800    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy" containerID="cri-o://cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190" gracePeriod=30
Sep 18 16:06:11.576756 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:11.576724    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 16:06:11.576965 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:11.576796    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 16:06:16.416946 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.416916    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:06:16.514662 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.514629    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-podinfo\") pod \"6498153f-ce82-4c89-971f-ee1f2d66bc94\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") "
Sep 18 16:06:16.514859 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.514673    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2h5x\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-kube-api-access-z2h5x\") pod \"6498153f-ce82-4c89-971f-ee1f2d66bc94\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") "
Sep 18 16:06:16.514859 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.514693    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6498153f-ce82-4c89-971f-ee1f2d66bc94-istiod-ca-cert\") pod \"6498153f-ce82-4c89-971f-ee1f2d66bc94\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") "
Sep 18 16:06:16.514859 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.514718    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-envoy\") pod \"6498153f-ce82-4c89-971f-ee1f2d66bc94\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") "
Sep 18 16:06:16.514859 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.514757    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-credential-socket\") pod \"6498153f-ce82-4c89-971f-ee1f2d66bc94\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") "
Sep 18 16:06:16.514859 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.514828    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-data\") pod \"6498153f-ce82-4c89-971f-ee1f2d66bc94\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") "
Sep 18 16:06:16.515152 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.514884    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-socket\") pod \"6498153f-ce82-4c89-971f-ee1f2d66bc94\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") "
Sep 18 16:06:16.515152 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.514931    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-certs\") pod \"6498153f-ce82-4c89-971f-ee1f2d66bc94\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") "
Sep 18 16:06:16.515152 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.514999    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-token\") pod \"6498153f-ce82-4c89-971f-ee1f2d66bc94\" (UID: \"6498153f-ce82-4c89-971f-ee1f2d66bc94\") "
Sep 18 16:06:16.515152 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515131    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6498153f-ce82-4c89-971f-ee1f2d66bc94-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6498153f-ce82-4c89-971f-ee1f2d66bc94" (UID: "6498153f-ce82-4c89-971f-ee1f2d66bc94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:06:16.515367 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515288    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6498153f-ce82-4c89-971f-ee1f2d66bc94" (UID: "6498153f-ce82-4c89-971f-ee1f2d66bc94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:16.515367 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515301    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6498153f-ce82-4c89-971f-ee1f2d66bc94" (UID: "6498153f-ce82-4c89-971f-ee1f2d66bc94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:16.515475 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515395    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6498153f-ce82-4c89-971f-ee1f2d66bc94" (UID: "6498153f-ce82-4c89-971f-ee1f2d66bc94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:16.515475 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515409    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-data" (OuterVolumeSpecName: "istio-data") pod "6498153f-ce82-4c89-971f-ee1f2d66bc94" (UID: "6498153f-ce82-4c89-971f-ee1f2d66bc94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:16.515638 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515614    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:16.515709 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515639    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:16.515709 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515653    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:16.515709 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515668    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6498153f-ce82-4c89-971f-ee1f2d66bc94-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:16.515709 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.515680    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:16.517393 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.517361    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-kube-api-access-z2h5x" (OuterVolumeSpecName: "kube-api-access-z2h5x") pod "6498153f-ce82-4c89-971f-ee1f2d66bc94" (UID: "6498153f-ce82-4c89-971f-ee1f2d66bc94"). InnerVolumeSpecName "kube-api-access-z2h5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:06:16.517523 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.517399    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6498153f-ce82-4c89-971f-ee1f2d66bc94" (UID: "6498153f-ce82-4c89-971f-ee1f2d66bc94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 16:06:16.517523 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.517399    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6498153f-ce82-4c89-971f-ee1f2d66bc94" (UID: "6498153f-ce82-4c89-971f-ee1f2d66bc94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:16.517722 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.517692    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-token" (OuterVolumeSpecName: "istio-token") pod "6498153f-ce82-4c89-971f-ee1f2d66bc94" (UID: "6498153f-ce82-4c89-971f-ee1f2d66bc94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:06:16.616827 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.616793    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:16.616827 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.616826    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2h5x\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-kube-api-access-z2h5x\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:16.617111 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.616866    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:16.617111 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:16.616879    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6498153f-ce82-4c89-971f-ee1f2d66bc94-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:17.243910 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.243872    2585 generic.go:356] "Generic (PLEG): container finished" podID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerID="cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190" exitCode=0
Sep 18 16:06:17.244108 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.243923    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"
Sep 18 16:06:17.244108 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.243960    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" event={"ID":"6498153f-ce82-4c89-971f-ee1f2d66bc94","Type":"ContainerDied","Data":"cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190"}
Sep 18 16:06:17.244108 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.244012    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd" event={"ID":"6498153f-ce82-4c89-971f-ee1f2d66bc94","Type":"ContainerDied","Data":"2d3bbfaf0e19beda169cbcd0e33e49925d0744937f48c831d5ad136c12b8b66e"}
Sep 18 16:06:17.244108 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.244033    2585 scope.go:117] "RemoveContainer" containerID="cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190"
Sep 18 16:06:17.253933 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.253915    2585 scope.go:117] "RemoveContainer" containerID="0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07"
Sep 18 16:06:17.262028 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.262012    2585 scope.go:117] "RemoveContainer" containerID="cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190"
Sep 18 16:06:17.262282 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:06:17.262261    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190\": container with ID starting with cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190 not found: ID does not exist" containerID="cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190"
Sep 18 16:06:17.262325 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.262292    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190"} err="failed to get container status \"cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190\": rpc error: code = NotFound desc = could not find container \"cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190\": container with ID starting with cb61ed7eadbdbc6dda474c93b04987a014251bab3312e0ab3c9a8f175865c190 not found: ID does not exist"
Sep 18 16:06:17.262325 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.262308    2585 scope.go:117] "RemoveContainer" containerID="0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07"
Sep 18 16:06:17.262538 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:06:17.262519    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07\": container with ID starting with 0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07 not found: ID does not exist" containerID="0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07"
Sep 18 16:06:17.262595 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.262546    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07"} err="failed to get container status \"0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07\": rpc error: code = NotFound desc = could not find container \"0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07\": container with ID starting with 0aaaec9874d170bad3395f3ce2cb4ac60593696708304cccad0656c365678c07 not found: ID does not exist"
Sep 18 16:06:17.268130 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.268107    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"]
Sep 18 16:06:17.274114 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.274090    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-e41875-openshift-default-55796d58f6-jlffd"]
Sep 18 16:06:17.523447 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:17.523351    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" path="/var/lib/kubelet/pods/6498153f-ce82-4c89-971f-ee1f2d66bc94/volumes"
Sep 18 16:06:28.924210 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:28.924170    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"]
Sep 18 16:06:28.924692 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:28.924375    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy" containerID="cri-o://74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e" gracePeriod=30
Sep 18 16:06:32.248462 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:32.248425    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 16:06:32.248883 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:32.248495    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 16:06:34.175599 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.175573    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 16:06:34.285146 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285111    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-credential-socket\") pod \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") "
Sep 18 16:06:34.285146 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285150    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istiod-ca-cert\") pod \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") "
Sep 18 16:06:34.285401 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285192    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-token\") pod \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") "
Sep 18 16:06:34.285401 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285219    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-socket\") pod \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") "
Sep 18 16:06:34.285401 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285244    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-data\") pod \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") "
Sep 18 16:06:34.285401 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285282    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-envoy\") pod \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") "
Sep 18 16:06:34.285401 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285351    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxck9\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-kube-api-access-qxck9\") pod \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") "
Sep 18 16:06:34.285401 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285398    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-certs\") pod \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") "
Sep 18 16:06:34.285733 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285405    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4dfc6a8-6f30-4936-9278-6190e1de24a5" (UID: "b4dfc6a8-6f30-4936-9278-6190e1de24a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:34.285733 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285422    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-podinfo\") pod \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\" (UID: \"b4dfc6a8-6f30-4936-9278-6190e1de24a5\") "
Sep 18 16:06:34.285733 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285552    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4dfc6a8-6f30-4936-9278-6190e1de24a5" (UID: "b4dfc6a8-6f30-4936-9278-6190e1de24a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:06:34.285733 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285687    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4dfc6a8-6f30-4936-9278-6190e1de24a5" (UID: "b4dfc6a8-6f30-4936-9278-6190e1de24a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:34.285922 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285807    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4dfc6a8-6f30-4936-9278-6190e1de24a5" (UID: "b4dfc6a8-6f30-4936-9278-6190e1de24a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:34.285922 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285874    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4dfc6a8-6f30-4936-9278-6190e1de24a5" (UID: "b4dfc6a8-6f30-4936-9278-6190e1de24a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:34.285922 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285891    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-credential-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:34.285922 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285912    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istiod-ca-cert\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:34.286149 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.285926    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-data\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:34.287523 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.287491    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4dfc6a8-6f30-4936-9278-6190e1de24a5" (UID: "b4dfc6a8-6f30-4936-9278-6190e1de24a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 16:06:34.287665 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.287525    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-kube-api-access-qxck9" (OuterVolumeSpecName: "kube-api-access-qxck9") pod "b4dfc6a8-6f30-4936-9278-6190e1de24a5" (UID: "b4dfc6a8-6f30-4936-9278-6190e1de24a5"). InnerVolumeSpecName "kube-api-access-qxck9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:06:34.287665 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.287558    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4dfc6a8-6f30-4936-9278-6190e1de24a5" (UID: "b4dfc6a8-6f30-4936-9278-6190e1de24a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:06:34.287665 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.287592    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4dfc6a8-6f30-4936-9278-6190e1de24a5" (UID: "b4dfc6a8-6f30-4936-9278-6190e1de24a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:06:34.345834 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.345802    2585 generic.go:356] "Generic (PLEG): container finished" podID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerID="74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e" exitCode=0
Sep 18 16:06:34.346023 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.345875    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"
Sep 18 16:06:34.346023 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.345905    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" event={"ID":"b4dfc6a8-6f30-4936-9278-6190e1de24a5","Type":"ContainerDied","Data":"74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e"}
Sep 18 16:06:34.346023 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.345964    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64" event={"ID":"b4dfc6a8-6f30-4936-9278-6190e1de24a5","Type":"ContainerDied","Data":"d473aa7a3dcad6d9a7fd54b2e83b43c72fc7ca4f7e6230d55ab69985101d1f2c"}
Sep 18 16:06:34.346023 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.345989    2585 scope.go:117] "RemoveContainer" containerID="74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e"
Sep 18 16:06:34.355334 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.355318    2585 scope.go:117] "RemoveContainer" containerID="bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d"
Sep 18 16:06:34.363185 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.363169    2585 scope.go:117] "RemoveContainer" containerID="74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e"
Sep 18 16:06:34.363463 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:06:34.363442    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e\": container with ID starting with 74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e not found: ID does not exist" containerID="74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e"
Sep 18 16:06:34.363511 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.363471    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e"} err="failed to get container status \"74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e\": rpc error: code = NotFound desc = could not find container \"74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e\": container with ID starting with 74fe09b8cda27bd1ad4d7dc03726808728664f1f810a665622c1c27a59e8936e not found: ID does not exist"
Sep 18 16:06:34.363511 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.363489    2585 scope.go:117] "RemoveContainer" containerID="bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d"
Sep 18 16:06:34.363697 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:06:34.363682    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d\": container with ID starting with bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d not found: ID does not exist" containerID="bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d"
Sep 18 16:06:34.363746 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.363701    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d"} err="failed to get container status \"bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d\": rpc error: code = NotFound desc = could not find container \"bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d\": container with ID starting with bdfae902c5cb2454a1c6c22363e167f2eece029875ae07ed6dd74cf23174f06d not found: ID does not exist"
Sep 18 16:06:34.372810 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.372789    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"]
Sep 18 16:06:34.378118 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.378064    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f14aba-openshift-default-67db9c8b9b-tjf64"]
Sep 18 16:06:34.387043 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.387024    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-token\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:34.387043 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.387044    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-socket\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:34.387182 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.387053    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-envoy\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:34.387182 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.387063    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxck9\" (UniqueName: \"kubernetes.io/projected/b4dfc6a8-6f30-4936-9278-6190e1de24a5-kube-api-access-qxck9\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:34.387182 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.387073    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc6a8-6f30-4936-9278-6190e1de24a5-workload-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:34.387182 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:34.387082    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4dfc6a8-6f30-4936-9278-6190e1de24a5-istio-podinfo\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:35.523050 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:35.523018    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" path="/var/lib/kubelet/pods/b4dfc6a8-6f30-4936-9278-6190e1de24a5/volumes"
Sep 18 16:06:37.720420 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:37.720394    2585 scope.go:117] "RemoveContainer" containerID="812d8535c02060937fe07676deaff402581e7e47b1f2bf70f06e8148751d687c"
Sep 18 16:06:39.834103 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:39.834058    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"]
Sep 18 16:06:39.835162 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:39.835100    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7" podUID="da985123-a3b8-4781-96b2-43484984ee22" containerName="maas-api" containerID="cri-o://3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e" gracePeriod=30
Sep 18 16:06:40.091797 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.091733    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 16:06:40.240733 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.240697    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-maas-api-tls\") pod \"da985123-a3b8-4781-96b2-43484984ee22\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") "
Sep 18 16:06:40.240733 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.240735    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da985123-a3b8-4781-96b2-43484984ee22-openshift-service-ca\") pod \"da985123-a3b8-4781-96b2-43484984ee22\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") "
Sep 18 16:06:40.241019 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.240768    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-metrics-certs\") pod \"da985123-a3b8-4781-96b2-43484984ee22\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") "
Sep 18 16:06:40.241019 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.240814    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9kb4\" (UniqueName: \"kubernetes.io/projected/da985123-a3b8-4781-96b2-43484984ee22-kube-api-access-s9kb4\") pod \"da985123-a3b8-4781-96b2-43484984ee22\" (UID: \"da985123-a3b8-4781-96b2-43484984ee22\") "
Sep 18 16:06:40.241206 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.241180    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da985123-a3b8-4781-96b2-43484984ee22-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da985123-a3b8-4781-96b2-43484984ee22" (UID: "da985123-a3b8-4781-96b2-43484984ee22"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:06:40.243610 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.243574    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "da985123-a3b8-4781-96b2-43484984ee22" (UID: "da985123-a3b8-4781-96b2-43484984ee22"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:06:40.244238 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.244175    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da985123-a3b8-4781-96b2-43484984ee22-kube-api-access-s9kb4" (OuterVolumeSpecName: "kube-api-access-s9kb4") pod "da985123-a3b8-4781-96b2-43484984ee22" (UID: "da985123-a3b8-4781-96b2-43484984ee22"). InnerVolumeSpecName "kube-api-access-s9kb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:06:40.245035 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.245009    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da985123-a3b8-4781-96b2-43484984ee22" (UID: "da985123-a3b8-4781-96b2-43484984ee22"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:06:40.342315 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.342226    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:40.342315 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.342257    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da985123-a3b8-4781-96b2-43484984ee22-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:40.342315 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.342267    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da985123-a3b8-4781-96b2-43484984ee22-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:40.342315 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.342276    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9kb4\" (UniqueName: \"kubernetes.io/projected/da985123-a3b8-4781-96b2-43484984ee22-kube-api-access-s9kb4\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:06:40.383998 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.383967    2585 generic.go:356] "Generic (PLEG): container finished" podID="da985123-a3b8-4781-96b2-43484984ee22" containerID="3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e" exitCode=0
Sep 18 16:06:40.384181 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.384011    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"
Sep 18 16:06:40.384181 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.384044    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7" event={"ID":"da985123-a3b8-4781-96b2-43484984ee22","Type":"ContainerDied","Data":"3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e"}
Sep 18 16:06:40.384181 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.384099    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7" event={"ID":"da985123-a3b8-4781-96b2-43484984ee22","Type":"ContainerDied","Data":"cae52a5cd91a74623639c185923bf2f745f346a94a8b260e1ffa7235334d7499"}
Sep 18 16:06:40.384181 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.384124    2585 scope.go:117] "RemoveContainer" containerID="3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e"
Sep 18 16:06:40.398103 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.398085    2585 scope.go:117] "RemoveContainer" containerID="3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e"
Sep 18 16:06:40.398375 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:06:40.398353    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e\": container with ID starting with 3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e not found: ID does not exist" containerID="3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e"
Sep 18 16:06:40.398438 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.398386    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e"} err="failed to get container status \"3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e\": rpc error: code = NotFound desc = could not find container \"3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e\": container with ID starting with 3ae3b77a7b8ac019a63edc08a3d5b7a60bb5b2e6d731bfd80bfd809fedee214e not found: ID does not exist"
Sep 18 16:06:40.408519 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.408495    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"]
Sep 18 16:06:40.417263 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:40.417239    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4e81aa-5449f96b5d-vtph7"]
Sep 18 16:06:41.524504 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:06:41.524469    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da985123-a3b8-4781-96b2-43484984ee22" path="/var/lib/kubelet/pods/da985123-a3b8-4781-96b2-43484984ee22/volumes"
Sep 18 16:09:23.750826 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:23.750795    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"]
Sep 18 16:09:23.751295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:23.751001    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd" podUID="b1061a8b-e582-4746-bbc6-1200b22fa438" containerName="manager" containerID="cri-o://deb3f9d7854ef3ab06d52f84698211f742d5bbfe271bd6f50ba56580ae79587d" gracePeriod=10
Sep 18 16:09:24.275094 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:24.275004    2585 generic.go:356] "Generic (PLEG): container finished" podID="b1061a8b-e582-4746-bbc6-1200b22fa438" containerID="deb3f9d7854ef3ab06d52f84698211f742d5bbfe271bd6f50ba56580ae79587d" exitCode=0
Sep 18 16:09:24.275094 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:24.275072    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd" event={"ID":"b1061a8b-e582-4746-bbc6-1200b22fa438","Type":"ContainerDied","Data":"deb3f9d7854ef3ab06d52f84698211f742d5bbfe271bd6f50ba56580ae79587d"}
Sep 18 16:09:24.646540 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:24.646516    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 16:09:24.768237 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:24.768208    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hbzz\" (UniqueName: \"kubernetes.io/projected/b1061a8b-e582-4746-bbc6-1200b22fa438-kube-api-access-2hbzz\") pod \"b1061a8b-e582-4746-bbc6-1200b22fa438\" (UID: \"b1061a8b-e582-4746-bbc6-1200b22fa438\") "
Sep 18 16:09:24.768690 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:24.768247    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1061a8b-e582-4746-bbc6-1200b22fa438-extensions-socket-volume\") pod \"b1061a8b-e582-4746-bbc6-1200b22fa438\" (UID: \"b1061a8b-e582-4746-bbc6-1200b22fa438\") "
Sep 18 16:09:24.768690 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:24.768663    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1061a8b-e582-4746-bbc6-1200b22fa438-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b1061a8b-e582-4746-bbc6-1200b22fa438" (UID: "b1061a8b-e582-4746-bbc6-1200b22fa438"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:09:24.770210 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:24.770183    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1061a8b-e582-4746-bbc6-1200b22fa438-kube-api-access-2hbzz" (OuterVolumeSpecName: "kube-api-access-2hbzz") pod "b1061a8b-e582-4746-bbc6-1200b22fa438" (UID: "b1061a8b-e582-4746-bbc6-1200b22fa438"). InnerVolumeSpecName "kube-api-access-2hbzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:09:24.869863 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:24.869777    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hbzz\" (UniqueName: \"kubernetes.io/projected/b1061a8b-e582-4746-bbc6-1200b22fa438-kube-api-access-2hbzz\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:09:24.869863 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:24.869806    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1061a8b-e582-4746-bbc6-1200b22fa438-extensions-socket-volume\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:09:25.281990 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:25.281961    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"
Sep 18 16:09:25.282186 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:25.281957    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd" event={"ID":"b1061a8b-e582-4746-bbc6-1200b22fa438","Type":"ContainerDied","Data":"9154a3966fbdd0488065219b65c06f2911bef3c66e2f4c3b6869a4649a103b80"}
Sep 18 16:09:25.282186 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:25.282094    2585 scope.go:117] "RemoveContainer" containerID="deb3f9d7854ef3ab06d52f84698211f742d5bbfe271bd6f50ba56580ae79587d"
Sep 18 16:09:25.308701 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:25.308678    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"]
Sep 18 16:09:25.313694 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:25.313673    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6hwrd"]
Sep 18 16:09:25.523444 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:25.523413    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1061a8b-e582-4746-bbc6-1200b22fa438" path="/var/lib/kubelet/pods/b1061a8b-e582-4746-bbc6-1200b22fa438/volumes"
Sep 18 16:09:37.671019 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:37.670980    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 16:09:37.675598 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:09:37.675577    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6wqf6_5a474076-cb9a-4eb3-8f05-c29502e12b38/ovn-acl-logging/0.log"
Sep 18 16:11:08.747121 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747084    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"]
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747629    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy"
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747650    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy"
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747668    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy"
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747678    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy"
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747690    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1061a8b-e582-4746-bbc6-1200b22fa438" containerName="manager"
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747700    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1061a8b-e582-4746-bbc6-1200b22fa438" containerName="manager"
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747709    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy"
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747718    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy"
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747733    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy"
Sep 18 16:11:08.747741 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747741    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747751    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747759    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747768    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da985123-a3b8-4781-96b2-43484984ee22" containerName="maas-api"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747776    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="da985123-a3b8-4781-96b2-43484984ee22" containerName="maas-api"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747787    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747796    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747917    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747931    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747940    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1061a8b-e582-4746-bbc6-1200b22fa438" containerName="manager"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747949    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="da985123-a3b8-4781-96b2-43484984ee22" containerName="maas-api"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747959    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747968    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6498153f-ce82-4c89-971f-ee1f2d66bc94" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747976    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy"
Sep 18 16:11:08.748295 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.747985    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0c7b919-7ac4-483c-a4a2-d21f5e9f3385" containerName="istio-proxy"
Sep 18 16:11:08.751424 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.751405    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.755380 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.755358    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-a0240115-kserve-self-signed-certs\""
Sep 18 16:11:08.808879 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.808826    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"]
Sep 18 16:11:08.845816 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.845789    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-tmp-dir\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.845966 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.845824    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-dshm\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.845966 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.845927    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-home\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.846047 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.845973    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-model-cache\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.846047 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.846001    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08f57a7f-274d-4d57-9d30-12dfef6ef292-tls-certs\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.846047 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.846019    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqx2b\" (UniqueName: \"kubernetes.io/projected/08f57a7f-274d-4d57-9d30-12dfef6ef292-kube-api-access-dqx2b\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.946851 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.946808    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-home\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.946994 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.946869    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-model-cache\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.946994 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.946897    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08f57a7f-274d-4d57-9d30-12dfef6ef292-tls-certs\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.946994 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.946915    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqx2b\" (UniqueName: \"kubernetes.io/projected/08f57a7f-274d-4d57-9d30-12dfef6ef292-kube-api-access-dqx2b\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.946994 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.946959    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-tmp-dir\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.947194 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.947006    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-dshm\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.947254 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.947236    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-model-cache\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.947310 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.947235    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-home\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.947410 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.947393    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-tmp-dir\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.949149 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.949127    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-dshm\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.949315 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.949298    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08f57a7f-274d-4d57-9d30-12dfef6ef292-tls-certs\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:08.961743 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:08.961722    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqx2b\" (UniqueName: \"kubernetes.io/projected/08f57a7f-274d-4d57-9d30-12dfef6ef292-kube-api-access-dqx2b\") pod \"e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") " pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:09.059995 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:09.059903    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:09.377433 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:09.377405    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"]
Sep 18 16:11:09.378464 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:11:09.378441    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08f57a7f_274d_4d57_9d30_12dfef6ef292.slice/crio-c2439ca2b5d582b1a47e7961f53391eb5c2fc2643b3f2ee5da87b709c8eb139f WatchSource:0}: Error finding container c2439ca2b5d582b1a47e7961f53391eb5c2fc2643b3f2ee5da87b709c8eb139f: Status 404 returned error can't find the container with id c2439ca2b5d582b1a47e7961f53391eb5c2fc2643b3f2ee5da87b709c8eb139f
Sep 18 16:11:09.842023 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:09.841986    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97" event={"ID":"08f57a7f-274d-4d57-9d30-12dfef6ef292","Type":"ContainerStarted","Data":"41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41"}
Sep 18 16:11:09.842432 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:09.842032    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97" event={"ID":"08f57a7f-274d-4d57-9d30-12dfef6ef292","Type":"ContainerStarted","Data":"c2439ca2b5d582b1a47e7961f53391eb5c2fc2643b3f2ee5da87b709c8eb139f"}
Sep 18 16:11:09.864075 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:09.864026    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97" podStartSLOduration=1.864009042 podStartE2EDuration="1.864009042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:11:09.862000128 +0000 UTC m=+2194.520792789" watchObservedRunningTime="2026-09-18 16:11:09.864009042 +0000 UTC m=+2194.522801702"
Sep 18 16:11:19.060186 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:19.060140    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:19.060584 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:19.060401    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:19.072585 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:19.072561    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:19.904205 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:19.904171    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:11:57.073874 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:11:57.073643    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"]
Sep 18 16:12:12.098977 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.098934    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97" podUID="08f57a7f-274d-4d57-9d30-12dfef6ef292" containerName="main" containerID="cri-o://41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41" gracePeriod=45
Sep 18 16:12:12.338830 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.338808    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:12:12.482703 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.482608    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-dshm\") pod \"08f57a7f-274d-4d57-9d30-12dfef6ef292\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") "
Sep 18 16:12:12.482703 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.482670    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08f57a7f-274d-4d57-9d30-12dfef6ef292-tls-certs\") pod \"08f57a7f-274d-4d57-9d30-12dfef6ef292\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") "
Sep 18 16:12:12.482703 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.482705    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqx2b\" (UniqueName: \"kubernetes.io/projected/08f57a7f-274d-4d57-9d30-12dfef6ef292-kube-api-access-dqx2b\") pod \"08f57a7f-274d-4d57-9d30-12dfef6ef292\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") "
Sep 18 16:12:12.483078 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.482730    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-model-cache\") pod \"08f57a7f-274d-4d57-9d30-12dfef6ef292\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") "
Sep 18 16:12:12.483078 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.482761    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-tmp-dir\") pod \"08f57a7f-274d-4d57-9d30-12dfef6ef292\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") "
Sep 18 16:12:12.483078 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.482935    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-home\") pod \"08f57a7f-274d-4d57-9d30-12dfef6ef292\" (UID: \"08f57a7f-274d-4d57-9d30-12dfef6ef292\") "
Sep 18 16:12:12.483215 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.483175    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-model-cache" (OuterVolumeSpecName: "model-cache") pod "08f57a7f-274d-4d57-9d30-12dfef6ef292" (UID: "08f57a7f-274d-4d57-9d30-12dfef6ef292"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:12:12.483274 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.483247    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-model-cache\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:12.483331 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.483288    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "08f57a7f-274d-4d57-9d30-12dfef6ef292" (UID: "08f57a7f-274d-4d57-9d30-12dfef6ef292"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:12:12.483331 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.483300    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-home" (OuterVolumeSpecName: "home") pod "08f57a7f-274d-4d57-9d30-12dfef6ef292" (UID: "08f57a7f-274d-4d57-9d30-12dfef6ef292"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:12:12.485113 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.485082    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-dshm" (OuterVolumeSpecName: "dshm") pod "08f57a7f-274d-4d57-9d30-12dfef6ef292" (UID: "08f57a7f-274d-4d57-9d30-12dfef6ef292"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 16:12:12.485229 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.485153    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08f57a7f-274d-4d57-9d30-12dfef6ef292-kube-api-access-dqx2b" (OuterVolumeSpecName: "kube-api-access-dqx2b") pod "08f57a7f-274d-4d57-9d30-12dfef6ef292" (UID: "08f57a7f-274d-4d57-9d30-12dfef6ef292"). InnerVolumeSpecName "kube-api-access-dqx2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:12:12.485229 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.485161    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08f57a7f-274d-4d57-9d30-12dfef6ef292-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "08f57a7f-274d-4d57-9d30-12dfef6ef292" (UID: "08f57a7f-274d-4d57-9d30-12dfef6ef292"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:12:12.584272 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.584244    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-dshm\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:12.584272 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.584269    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08f57a7f-274d-4d57-9d30-12dfef6ef292-tls-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:12.584446 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.584280    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqx2b\" (UniqueName: \"kubernetes.io/projected/08f57a7f-274d-4d57-9d30-12dfef6ef292-kube-api-access-dqx2b\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:12.584446 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.584289    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-tmp-dir\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:12.584446 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:12.584297    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08f57a7f-274d-4d57-9d30-12dfef6ef292-home\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:13.181772 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.181732    2585 generic.go:356] "Generic (PLEG): container finished" podID="08f57a7f-274d-4d57-9d30-12dfef6ef292" containerID="41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41" exitCode=0
Sep 18 16:12:13.182217 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.181784    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"
Sep 18 16:12:13.182217 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.181783    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97" event={"ID":"08f57a7f-274d-4d57-9d30-12dfef6ef292","Type":"ContainerDied","Data":"41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41"}
Sep 18 16:12:13.182217 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.181821    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97" event={"ID":"08f57a7f-274d-4d57-9d30-12dfef6ef292","Type":"ContainerDied","Data":"c2439ca2b5d582b1a47e7961f53391eb5c2fc2643b3f2ee5da87b709c8eb139f"}
Sep 18 16:12:13.182217 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.181865    2585 scope.go:117] "RemoveContainer" containerID="41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41"
Sep 18 16:12:13.191925 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.191899    2585 scope.go:117] "RemoveContainer" containerID="41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41"
Sep 18 16:12:13.192195 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:12:13.192171    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41\": container with ID starting with 41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41 not found: ID does not exist" containerID="41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41"
Sep 18 16:12:13.192290 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.192200    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41"} err="failed to get container status \"41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41\": rpc error: code = NotFound desc = could not find container \"41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41\": container with ID starting with 41ba583ae928973dbfe4c7b65d5fa63e20443f3f4013040749d64403e882cf41 not found: ID does not exist"
Sep 18 16:12:13.205023 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.205000    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"]
Sep 18 16:12:13.208933 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.208911    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-a0240115-kserve-6f7ff7bfb7-kvn97"]
Sep 18 16:12:13.523422 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:13.523384    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08f57a7f-274d-4d57-9d30-12dfef6ef292" path="/var/lib/kubelet/pods/08f57a7f-274d-4d57-9d30-12dfef6ef292/volumes"
Sep 18 16:12:15.212760 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.212729    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"]
Sep 18 16:12:15.213157 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.213114    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy"
Sep 18 16:12:15.213157 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.213127    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy"
Sep 18 16:12:15.213157 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.213143    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy"
Sep 18 16:12:15.213157 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.213148    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy"
Sep 18 16:12:15.213307 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.213161    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08f57a7f-274d-4d57-9d30-12dfef6ef292" containerName="main"
Sep 18 16:12:15.213307 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.213166    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="08f57a7f-274d-4d57-9d30-12dfef6ef292" containerName="main"
Sep 18 16:12:15.213307 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.213217    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4dfc6a8-6f30-4936-9278-6190e1de24a5" containerName="istio-proxy"
Sep 18 16:12:15.213307 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.213241    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c09bec6-33e1-4349-9907-e92523ad1367" containerName="istio-proxy"
Sep 18 16:12:15.213307 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.213249    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="08f57a7f-274d-4d57-9d30-12dfef6ef292" containerName="main"
Sep 18 16:12:15.217863 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.217828    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.231922 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.231898    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"]
Sep 18 16:12:15.307966 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.307930    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-metrics-certs\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.307966 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.307967    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-maas-api-tls\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.308220 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.307997    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-openshift-service-ca\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.308220 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.308157    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbxbz\" (UniqueName: \"kubernetes.io/projected/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-kube-api-access-kbxbz\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.409609 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.409558    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-metrics-certs\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.409795 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.409614    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-maas-api-tls\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.409795 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.409714    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-openshift-service-ca\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.409980 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.409811    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbxbz\" (UniqueName: \"kubernetes.io/projected/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-kube-api-access-kbxbz\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.410334 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.410312    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-openshift-service-ca\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.412104 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.412083    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-metrics-certs\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.412198 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.412106    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-maas-api-tls\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.418525 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.418503    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbxbz\" (UniqueName: \"kubernetes.io/projected/b2ba29ce-9931-4b0d-a9b1-60fba37a8f07-kube-api-access-kbxbz\") pod \"maas-api-86d9ddc875-sf7q6\" (UID: \"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.527517 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.527489    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:15.803482 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.803448    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"]
Sep 18 16:12:15.807249 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:12:15.807203    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ba29ce_9931_4b0d_a9b1_60fba37a8f07.slice/crio-cde146d4a6741e8511de3f3ab14598d52b97ef624da6b4f812ad7d4bebcf4cea WatchSource:0}: Error finding container cde146d4a6741e8511de3f3ab14598d52b97ef624da6b4f812ad7d4bebcf4cea: Status 404 returned error can't find the container with id cde146d4a6741e8511de3f3ab14598d52b97ef624da6b4f812ad7d4bebcf4cea
Sep 18 16:12:15.808899 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:15.808825    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 16:12:16.206188 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:16.206161    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6" event={"ID":"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07","Type":"ContainerStarted","Data":"cde146d4a6741e8511de3f3ab14598d52b97ef624da6b4f812ad7d4bebcf4cea"}
Sep 18 16:12:17.213019 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:17.212982    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6" event={"ID":"b2ba29ce-9931-4b0d-a9b1-60fba37a8f07","Type":"ContainerStarted","Data":"d13c0ac1204c72f42c123a9ae9d0e70bd0795f8ccaad4f88012d8b5595ae44cf"}
Sep 18 16:12:17.213420 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:17.213115    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:17.238908 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:17.238861    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6" podStartSLOduration=1.851230664 podStartE2EDuration="2.23883057s" totalImagesPullingTime="387.599906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:12:17.237216732 +0000 UTC m=+2261.896009392" watchObservedRunningTime="2026-09-18 16:12:17.23883057 +0000 UTC m=+2261.897623251"
Sep 18 16:12:23.222927 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:23.222900    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-sf7q6"
Sep 18 16:12:23.305323 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:23.305293    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"]
Sep 18 16:12:23.305529 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:23.305505    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg" podUID="56059caa-5300-4bcf-90a1-e2f782a7aecc" containerName="maas-api" containerID="cri-o://d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215" gracePeriod=30
Sep 18 16:12:24.544855 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.544809    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 16:12:24.691189 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.691104    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b52qv\" (UniqueName: \"kubernetes.io/projected/56059caa-5300-4bcf-90a1-e2f782a7aecc-kube-api-access-b52qv\") pod \"56059caa-5300-4bcf-90a1-e2f782a7aecc\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") "
Sep 18 16:12:24.691189 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.691153    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-metrics-certs\") pod \"56059caa-5300-4bcf-90a1-e2f782a7aecc\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") "
Sep 18 16:12:24.691424 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.691255    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-maas-api-tls\") pod \"56059caa-5300-4bcf-90a1-e2f782a7aecc\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") "
Sep 18 16:12:24.691424 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.691362    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56059caa-5300-4bcf-90a1-e2f782a7aecc-openshift-service-ca\") pod \"56059caa-5300-4bcf-90a1-e2f782a7aecc\" (UID: \"56059caa-5300-4bcf-90a1-e2f782a7aecc\") "
Sep 18 16:12:24.691748 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.691722    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56059caa-5300-4bcf-90a1-e2f782a7aecc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "56059caa-5300-4bcf-90a1-e2f782a7aecc" (UID: "56059caa-5300-4bcf-90a1-e2f782a7aecc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 16:12:24.693153 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.693130    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "56059caa-5300-4bcf-90a1-e2f782a7aecc" (UID: "56059caa-5300-4bcf-90a1-e2f782a7aecc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:12:24.693602 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.693579    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "56059caa-5300-4bcf-90a1-e2f782a7aecc" (UID: "56059caa-5300-4bcf-90a1-e2f782a7aecc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 16:12:24.693697 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.693607    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56059caa-5300-4bcf-90a1-e2f782a7aecc-kube-api-access-b52qv" (OuterVolumeSpecName: "kube-api-access-b52qv") pod "56059caa-5300-4bcf-90a1-e2f782a7aecc" (UID: "56059caa-5300-4bcf-90a1-e2f782a7aecc"). InnerVolumeSpecName "kube-api-access-b52qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 16:12:24.792296 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.792262    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56059caa-5300-4bcf-90a1-e2f782a7aecc-openshift-service-ca\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:24.792296 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.792290    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b52qv\" (UniqueName: \"kubernetes.io/projected/56059caa-5300-4bcf-90a1-e2f782a7aecc-kube-api-access-b52qv\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:24.792296 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.792301    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-metrics-certs\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:24.792506 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:24.792309    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56059caa-5300-4bcf-90a1-e2f782a7aecc-maas-api-tls\") on node \"ip-10-0-130-208.ec2.internal\" DevicePath \"\""
Sep 18 16:12:25.256990 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.256957    2585 generic.go:356] "Generic (PLEG): container finished" podID="56059caa-5300-4bcf-90a1-e2f782a7aecc" containerID="d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215" exitCode=0
Sep 18 16:12:25.257176 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.257002    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"
Sep 18 16:12:25.257176 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.257055    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg" event={"ID":"56059caa-5300-4bcf-90a1-e2f782a7aecc","Type":"ContainerDied","Data":"d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215"}
Sep 18 16:12:25.257176 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.257110    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg" event={"ID":"56059caa-5300-4bcf-90a1-e2f782a7aecc","Type":"ContainerDied","Data":"8cfe26172d34d136cb6e83204d56d9de1d53fe5d29283569bf8c9240035c0466"}
Sep 18 16:12:25.257176 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.257131    2585 scope.go:117] "RemoveContainer" containerID="d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215"
Sep 18 16:12:25.267128 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.267109    2585 scope.go:117] "RemoveContainer" containerID="d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215"
Sep 18 16:12:25.267404 ip-10-0-130-208 kubenswrapper[2585]: E0918 16:12:25.267385    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215\": container with ID starting with d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215 not found: ID does not exist" containerID="d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215"
Sep 18 16:12:25.267480 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.267411    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215"} err="failed to get container status \"d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215\": rpc error: code = NotFound desc = could not find container \"d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215\": container with ID starting with d5882cf5dfb2934f5da6f21f12da8fe734c9a509ec0dac5b266eb850e8552215 not found: ID does not exist"
Sep 18 16:12:25.280647 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.280618    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"]
Sep 18 16:12:25.285697 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.285674    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg"]
Sep 18 16:12:25.522991 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:25.522909    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56059caa-5300-4bcf-90a1-e2f782a7aecc" path="/var/lib/kubelet/pods/56059caa-5300-4bcf-90a1-e2f782a7aecc/volumes"
Sep 18 16:12:27.498811 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:27.498749    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-ppmjg" podUID="56059caa-5300-4bcf-90a1-e2f782a7aecc" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.44:8443/health\": context deadline exceeded"
Sep 18 16:12:49.531158 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.531122    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2zs2w/must-gather-rm7c5"]
Sep 18 16:12:49.531530 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.531441    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56059caa-5300-4bcf-90a1-e2f782a7aecc" containerName="maas-api"
Sep 18 16:12:49.531530 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.531452    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="56059caa-5300-4bcf-90a1-e2f782a7aecc" containerName="maas-api"
Sep 18 16:12:49.531530 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.531522    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="56059caa-5300-4bcf-90a1-e2f782a7aecc" containerName="maas-api"
Sep 18 16:12:49.536156 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.536139    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2zs2w/must-gather-rm7c5"
Sep 18 16:12:49.539169 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.539146    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2zs2w\"/\"openshift-service-ca.crt\""
Sep 18 16:12:49.539313 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.539296    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2zs2w\"/\"default-dockercfg-k85dh\""
Sep 18 16:12:49.539510 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.539488    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2zs2w\"/\"kube-root-ca.crt\""
Sep 18 16:12:49.555676 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.555653    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2zs2w/must-gather-rm7c5"]
Sep 18 16:12:49.618572 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.618533    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzxg6\" (UniqueName: \"kubernetes.io/projected/20b9f5f4-fe51-469c-8d16-b38e49f63581-kube-api-access-kzxg6\") pod \"must-gather-rm7c5\" (UID: \"20b9f5f4-fe51-469c-8d16-b38e49f63581\") " pod="openshift-must-gather-2zs2w/must-gather-rm7c5"
Sep 18 16:12:49.618572 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.618574    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20b9f5f4-fe51-469c-8d16-b38e49f63581-must-gather-output\") pod \"must-gather-rm7c5\" (UID: \"20b9f5f4-fe51-469c-8d16-b38e49f63581\") " pod="openshift-must-gather-2zs2w/must-gather-rm7c5"
Sep 18 16:12:49.719487 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.719440    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzxg6\" (UniqueName: \"kubernetes.io/projected/20b9f5f4-fe51-469c-8d16-b38e49f63581-kube-api-access-kzxg6\") pod \"must-gather-rm7c5\" (UID: \"20b9f5f4-fe51-469c-8d16-b38e49f63581\") " pod="openshift-must-gather-2zs2w/must-gather-rm7c5"
Sep 18 16:12:49.719487 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.719488    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20b9f5f4-fe51-469c-8d16-b38e49f63581-must-gather-output\") pod \"must-gather-rm7c5\" (UID: \"20b9f5f4-fe51-469c-8d16-b38e49f63581\") " pod="openshift-must-gather-2zs2w/must-gather-rm7c5"
Sep 18 16:12:49.719780 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.719764    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20b9f5f4-fe51-469c-8d16-b38e49f63581-must-gather-output\") pod \"must-gather-rm7c5\" (UID: \"20b9f5f4-fe51-469c-8d16-b38e49f63581\") " pod="openshift-must-gather-2zs2w/must-gather-rm7c5"
Sep 18 16:12:49.728537 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.728506    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzxg6\" (UniqueName: \"kubernetes.io/projected/20b9f5f4-fe51-469c-8d16-b38e49f63581-kube-api-access-kzxg6\") pod \"must-gather-rm7c5\" (UID: \"20b9f5f4-fe51-469c-8d16-b38e49f63581\") " pod="openshift-must-gather-2zs2w/must-gather-rm7c5"
Sep 18 16:12:49.845411 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:49.845323    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2zs2w/must-gather-rm7c5"
Sep 18 16:12:50.117602 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:50.117574    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2zs2w/must-gather-rm7c5"]
Sep 18 16:12:50.118551 ip-10-0-130-208 kubenswrapper[2585]: W0918 16:12:50.118525    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20b9f5f4_fe51_469c_8d16_b38e49f63581.slice/crio-330d9217c606fd7db0023296261374f6d79a0fff959a91e4e1760ded40ab3900 WatchSource:0}: Error finding container 330d9217c606fd7db0023296261374f6d79a0fff959a91e4e1760ded40ab3900: Status 404 returned error can't find the container with id 330d9217c606fd7db0023296261374f6d79a0fff959a91e4e1760ded40ab3900
Sep 18 16:12:50.391744 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:50.391651    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2zs2w/must-gather-rm7c5" event={"ID":"20b9f5f4-fe51-469c-8d16-b38e49f63581","Type":"ContainerStarted","Data":"330d9217c606fd7db0023296261374f6d79a0fff959a91e4e1760ded40ab3900"}
Sep 18 16:12:51.400288 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:51.400256    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2zs2w/must-gather-rm7c5" event={"ID":"20b9f5f4-fe51-469c-8d16-b38e49f63581","Type":"ContainerStarted","Data":"726f12dfd49543608220d0896ffda489d27c887787cdae1b4427e764c1750382"}
Sep 18 16:12:51.400718 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:51.400298    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2zs2w/must-gather-rm7c5" event={"ID":"20b9f5f4-fe51-469c-8d16-b38e49f63581","Type":"ContainerStarted","Data":"f2ce6ed37462c43424d2b377f8afbacee3813b8f8818b593127f2c293197c4ec"}
Sep 18 16:12:51.421467 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:51.421399    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2zs2w/must-gather-rm7c5" podStartSLOduration=1.68126965 podStartE2EDuration="2.421378055s" totalImagesPullingTime="740.108405ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:12:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:12:51.416775261 +0000 UTC m=+2296.075567922" watchObservedRunningTime="2026-09-18 16:12:51.421378055 +0000 UTC m=+2296.080170717"
Sep 18 16:12:52.703256 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:52.703224    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bvgpr_f3967c88-e022-44a2-919c-468d1f757435/global-pull-secret-syncer/0.log"
Sep 18 16:12:53.066791 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:53.066760    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dl59b_ae988a2f-8f5c-4506-a5ba-ba8d7d79212e/konnectivity-agent/0.log"
Sep 18 16:12:53.199962 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:53.199930    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-208.ec2.internal_aa0a25f1f0b432907b52509419f94e73/haproxy/0.log"
Sep 18 16:12:56.612613 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:56.612577    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-mwxbq_5720ae4e-58b4-445c-ba0b-dafef88e6c0e/manager/0.log"
Sep 18 16:12:58.868525 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:58.868495    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f4c545679-fn6nc_c776300e-fe76-4bd8-9d60-a4ead7aff655/metrics-server/0.log"
Sep 18 16:12:58.899025 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:58.898996    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-nw59m_74b7e6bb-9cbc-4cca-aef5-f0f9721cfa38/monitoring-plugin/0.log"
Sep 18 16:12:59.124939 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:59.124824    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xlpxh_a9367d11-2046-437f-b049-0f496b2835f0/node-exporter/0.log"
Sep 18 16:12:59.152618 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:59.152594    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xlpxh_a9367d11-2046-437f-b049-0f496b2835f0/kube-rbac-proxy/0.log"
Sep 18 16:12:59.180458 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:59.180425    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xlpxh_a9367d11-2046-437f-b049-0f496b2835f0/init-textfile/0.log"
Sep 18 16:12:59.231441 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:59.231416    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-rgkn6_474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d/kube-rbac-proxy-main/0.log"
Sep 18 16:12:59.259145 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:59.259113    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-rgkn6_474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d/kube-rbac-proxy-self/0.log"
Sep 18 16:12:59.288444 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:59.288408    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-rgkn6_474c6af7-f61b-4f0c-b1f5-1fe06ca7cf8d/openshift-state-metrics/0.log"
Sep 18 16:12:59.542118 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:59.542087    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-rmq6h_a2108e61-95bf-4790-8073-7254ded185d6/prometheus-operator/0.log"
Sep 18 16:12:59.565125 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:59.565096    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-rmq6h_a2108e61-95bf-4790-8073-7254ded185d6/kube-rbac-proxy/0.log"
Sep 18 16:12:59.594890 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:12:59.594863    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-8cbfj_e6105c12-ac05-4aa8-b9d2-2ec4f4ff723a/prometheus-operator-admission-webhook/0.log"
Sep 18 16:13:03.174251 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.174216    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"]
Sep 18 16:13:03.179758 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.179723    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.188096 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.188033    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"]
Sep 18 16:13:03.270304 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.270258    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-sys\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.270509 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.270319    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-lib-modules\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.270509 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.270374    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-proc\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.270509 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.270407    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-podres\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.270509 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.270438    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ddl4\" (UniqueName: \"kubernetes.io/projected/b2db6247-16ce-4050-9966-f1e3fbda9f77-kube-api-access-7ddl4\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.371403 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.371360    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-sys\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.371606 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.371417    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-lib-modules\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.371606 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.371472    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-proc\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.371606 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.371482    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-sys\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.371606 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.371517    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-podres\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.371606 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.371545    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-proc\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.371606 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.371554    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ddl4\" (UniqueName: \"kubernetes.io/projected/b2db6247-16ce-4050-9966-f1e3fbda9f77-kube-api-access-7ddl4\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.371606 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.371577    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-lib-modules\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.371990 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.371637    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2db6247-16ce-4050-9966-f1e3fbda9f77-podres\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.382272 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.382243    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ddl4\" (UniqueName: \"kubernetes.io/projected/b2db6247-16ce-4050-9966-f1e3fbda9f77-kube-api-access-7ddl4\") pod \"perf-node-gather-daemonset-gzksl\" (UID: \"b2db6247-16ce-4050-9966-f1e3fbda9f77\") " pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.494265 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.494236    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:03.808375 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:03.808184    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"]
Sep 18 16:13:04.524018 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:04.523969    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl" event={"ID":"b2db6247-16ce-4050-9966-f1e3fbda9f77","Type":"ContainerStarted","Data":"cec79ad7a7383eed3cc3f2de890d3ccd614b443b7cff630bcd1743a5e6186dea"}
Sep 18 16:13:04.524421 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:04.524033    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl" event={"ID":"b2db6247-16ce-4050-9966-f1e3fbda9f77","Type":"ContainerStarted","Data":"f83ca6dac1325fd472b18ae65b1c17989031dafe1840e1f6172abe7e9902572e"}
Sep 18 16:13:04.524421 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:04.524170    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:04.543496 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:04.543440    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl" podStartSLOduration=1.543420059 podStartE2EDuration="1.543420059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:13:04.542526586 +0000 UTC m=+2309.201319247" watchObservedRunningTime="2026-09-18 16:13:04.543420059 +0000 UTC m=+2309.202212720"
Sep 18 16:13:04.898599 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:04.898521    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mjcsb_39f429b6-f95f-4ebe-8b50-d31ad9030a07/dns/0.log"
Sep 18 16:13:04.922528 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:04.922504    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mjcsb_39f429b6-f95f-4ebe-8b50-d31ad9030a07/kube-rbac-proxy/0.log"
Sep 18 16:13:05.057116 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:05.057091    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rnzfc_b73dcb9d-63e4-4ca8-9d26-9c3674c3e6d1/dns-node-resolver/0.log"
Sep 18 16:13:05.692096 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:05.692071    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hhm7w_9eb2cbd7-9c45-4158-88f1-39cc90d66fc4/node-ca/0.log"
Sep 18 16:13:06.691928 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:06.691896    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48_3ecb7141-4a46-45f5-ad03-18f347393040/istio-proxy/0.log"
Sep 18 16:13:07.141034 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:07.140998    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k589d_d22ec279-6ea8-49f0-9524-548a030ef448/discovery/0.log"
Sep 18 16:13:07.164538 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:07.164511    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fbb59494c-8wkbn_7a82478e-701b-45ba-9d23-4eb836cc9f52/kube-auth-proxy/0.log"
Sep 18 16:13:07.230164 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:07.230135    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6prf9_9502d995-c7d8-46e9-b1d0-5342b23810d5/istio-proxy/0.log"
Sep 18 16:13:07.233448 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:07.233413    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6prf9_9502d995-c7d8-46e9-b1d0-5342b23810d5/istio-proxy/1.log"
Sep 18 16:13:08.270200 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:08.270172    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8vxph_512287cb-bcbb-47c2-b1b7-c4d0d516d227/serve-healthcheck-canary/0.log"
Sep 18 16:13:09.178739 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:09.178676    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ft4rq_5c9f219a-1495-4ca8-b6c1-863efb7c7140/kube-rbac-proxy/0.log"
Sep 18 16:13:09.213543 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:09.213513    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ft4rq_5c9f219a-1495-4ca8-b6c1-863efb7c7140/exporter/0.log"
Sep 18 16:13:09.261751 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:09.261720    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ft4rq_5c9f219a-1495-4ca8-b6c1-863efb7c7140/extractor/0.log"
Sep 18 16:13:10.541459 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:10.541425    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2zs2w/perf-node-gather-daemonset-gzksl"
Sep 18 16:13:11.944925 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:11.944888    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wxcsp_7ca7bdbe-7481-4158-ad65-cdbdd278a354/server/0.log"
Sep 18 16:13:12.047907 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:12.046871    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lsph5_e10a9b66-543d-42eb-a912-7d1efeb41da1/manager/0.log"
Sep 18 16:13:12.074800 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:12.074771    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54b8c8d665-68hsx_73c5053f-5d25-4dc2-83b8-62feafa2ead4/manager/0.log"
Sep 18 16:13:12.110522 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:12.110499    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54b8c8d665-68hsx_73c5053f-5d25-4dc2-83b8-62feafa2ead4/copy-manifests/0.log"
Sep 18 16:13:13.062129 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:13.062080    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48_3ecb7141-4a46-45f5-ad03-18f347393040/istio-proxy/0.log"
Sep 18 16:13:13.524866 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:13.524819    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k589d_d22ec279-6ea8-49f0-9524-548a030ef448/discovery/0.log"
Sep 18 16:13:13.544615 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:13.544589    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fbb59494c-8wkbn_7a82478e-701b-45ba-9d23-4eb836cc9f52/kube-auth-proxy/0.log"
Sep 18 16:13:13.580215 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:13.580183    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6prf9_9502d995-c7d8-46e9-b1d0-5342b23810d5/istio-proxy/0.log"
Sep 18 16:13:13.583642 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:13.583613    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6prf9_9502d995-c7d8-46e9-b1d0-5342b23810d5/istio-proxy/1.log"
Sep 18 16:13:13.859243 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:13.859216    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-5qh2g_352c8084-4ba7-4d0e-b17c-b419788ea8c6/openshift-lws-operator/0.log"
Sep 18 16:13:15.218559 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:15.218497    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-j7c48_3ecb7141-4a46-45f5-ad03-18f347393040/istio-proxy/0.log"
Sep 18 16:13:15.664535 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:15.664451    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k589d_d22ec279-6ea8-49f0-9524-548a030ef448/discovery/0.log"
Sep 18 16:13:15.688585 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:15.688558    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fbb59494c-8wkbn_7a82478e-701b-45ba-9d23-4eb836cc9f52/kube-auth-proxy/0.log"
Sep 18 16:13:15.737477 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:15.737451    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6prf9_9502d995-c7d8-46e9-b1d0-5342b23810d5/istio-proxy/0.log"
Sep 18 16:13:15.740767 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:15.740730    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6prf9_9502d995-c7d8-46e9-b1d0-5342b23810d5/istio-proxy/1.log"
Sep 18 16:13:20.620333 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:20.620261    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddhnl_6a044247-0dee-476f-b2e4-309e4e0982d8/kube-multus-additional-cni-plugins/0.log"
Sep 18 16:13:20.652050 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:20.652025    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddhnl_6a044247-0dee-476f-b2e4-309e4e0982d8/egress-router-binary-copy/0.log"
Sep 18 16:13:20.683283 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:20.683257    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddhnl_6a044247-0dee-476f-b2e4-309e4e0982d8/cni-plugins/0.log"
Sep 18 16:13:20.708772 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:20.708735    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddhnl_6a044247-0dee-476f-b2e4-309e4e0982d8/bond-cni-plugin/0.log"
Sep 18 16:13:20.732316 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:20.732228    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddhnl_6a044247-0dee-476f-b2e4-309e4e0982d8/routeoverride-cni/0.log"
Sep 18 16:13:20.760075 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:20.760048    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddhnl_6a044247-0dee-476f-b2e4-309e4e0982d8/whereabouts-cni-bincopy/0.log"
Sep 18 16:13:20.790054 ip-10-0-130-208 kubenswrapper[2585]: I0918 16:13:20.790019    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddhnl_6a044247-0dee-476f-b2e4-309e4e0982d8/whereabouts-cni/0.log"