Sep 15 22:27:40.121442 ip-10-0-128-247 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 22:27:40.531548 ip-10-0-128-247 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 22:27:40.531548 ip-10-0-128-247 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 22:27:40.531548 ip-10-0-128-247 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 22:27:40.531548 ip-10-0-128-247 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 22:27:40.531548 ip-10-0-128-247 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 22:27:40.533762 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.533660    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 22:27:40.539129 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539084    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:40.539129 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539123    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:40.539129 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539128    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:40.539129 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539131    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:40.539129 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539134    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:40.539129 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539137    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:40.539129 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539141    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539144    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539147    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539151    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539154    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539156    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539159    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539162    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539165    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539168    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539171    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539174    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539176    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539179    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539181    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539184    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539187    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539189    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539192    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539194    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:40.539391 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539197    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539200    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539202    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539205    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539207    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539213    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539216    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539218    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539221    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539223    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539226    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539229    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539232    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539235    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539238    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539241    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539244    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539247    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539249    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:40.539866 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539252    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539258    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539261    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539265    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539268    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539270    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539273    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539275    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539279    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539283    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539286    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539289    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539292    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539295    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539297    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539300    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539302    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539305    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539309    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:40.540349 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539312    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539315    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539317    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539320    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539322    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539325    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539327    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539331    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539334    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539337    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539340    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539343    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539346    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539349    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539353    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539357    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539360    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539363    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539366    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539369    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:40.540813 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539372    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539374    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539377    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539379    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539382    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539385    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539387    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539390    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539392    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539395    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539398    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539401    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539404    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539407    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:40.541392 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.539409    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:40.542211 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542185    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:40.542211 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542210    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:40.542211 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542214    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:40.542211 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542217    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:40.542211 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542221    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542225    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542228    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542231    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542234    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542237    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542239    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542242    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542244    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542247    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542249    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542252    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542256    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542259    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542262    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542264    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542267    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542270    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542273    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542275    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:40.542358 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542279    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542281    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542284    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542286    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542289    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542291    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542294    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542296    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542301    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542304    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542307    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542311    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542313    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542316    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542320    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542323    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542326    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542329    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542332    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:40.542849 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542335    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542338    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542340    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542343    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542346    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542349    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542352    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542356    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542358    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542361    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542364    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542367    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542369    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542372    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542376    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542380    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542383    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542386    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542388    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542391    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:40.543322 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542393    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542396    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542399    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542402    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542405    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542408    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542411    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542414    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542417    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542421    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542423    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542426    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542429    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542432    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542434    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542437    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542439    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542442    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542446    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:40.543817 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542449    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542453    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542456    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542458    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542461    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542463    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542466    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542469    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542472    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542474    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542477    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542480    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542483    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542485    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542488    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542490    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542493    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542887    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542893    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542895    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:40.544304 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542899    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542901    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542904    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542907    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542910    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542912    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542917    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542921    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542924    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542927    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542929    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542932    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542934    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542938    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542941    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542943    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542946    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542948    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542951    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:40.544818 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542954    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542957    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542960    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542963    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542967    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542971    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542973    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542976    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542979    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542981    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542984    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542987    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542990    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542993    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542995    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.542998    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543000    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543003    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543006    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543008    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:40.545310 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543011    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543013    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543016    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543019    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543021    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543025    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543028    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543031    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543034    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543036    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543039    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543042    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543044    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543047    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543049    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543052    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543055    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543058    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543060    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543063    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:40.545800 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543066    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543068    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543071    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543074    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543076    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543082    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543085    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543087    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543103    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543108    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543112    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543117    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543120    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543123    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543126    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543129    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543131    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543134    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543136    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543138    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:40.546316 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543141    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543144    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543147    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543149    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543151    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543154    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543157    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543160    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543162    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543165    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543168    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543171    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543174    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543177    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543180    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543183    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543185    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543267    2578 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543309    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543317    2578 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 22:27:40.546806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543321    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543326    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543329    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543333    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543339    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543342    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543345    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543349    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543353    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543356    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543359    2578 flags.go:64] FLAG: --cgroup-root=""
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543362    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543365    2578 flags.go:64] FLAG: --client-ca-file=""
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543369    2578 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543372    2578 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543377    2578 flags.go:64] FLAG: --cluster-domain=""
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543380    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543383    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543386    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543390    2578 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543394    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543397    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543401    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 22:27:40.547304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543405    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543408    2578 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543411    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543414    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543417    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543421    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543426    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543429    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543432    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543435    2578 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543438    2578 flags.go:64] FLAG: --enable-server="true"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543441    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543446    2578 flags.go:64] FLAG: --event-burst="100"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543449    2578 flags.go:64] FLAG: --event-qps="50"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543452    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543455    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543458    2578 flags.go:64] FLAG: --eviction-hard=""
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543461    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543464    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543467    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543470    2578 flags.go:64] FLAG: --eviction-soft=""
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543473    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543476    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543480    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543483    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 22:27:40.547883 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543486    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543488    2578 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543492    2578 flags.go:64] FLAG: --feature-gates=""
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543496    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543499    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543502    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543505    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543509    2578 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543511    2578 flags.go:64] FLAG: --help="false"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543514    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-128-247.ec2.internal"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543517    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543520    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543523    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543527    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543530    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543533    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543536    2578 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543539    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543542    2578 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543544    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543548    2578 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543550    2578 flags.go:64] FLAG: --kube-reserved=""
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543553    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543556    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 22:27:40.548512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543559    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543562    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543564    2578 flags.go:64] FLAG: --lock-file=""
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543567    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543570    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543573    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543578    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543582    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543585    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543587    2578 flags.go:64] FLAG: --logging-format="text"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543590    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543597    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543600    2578 flags.go:64] FLAG: --manifest-url=""
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543603    2578 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543612    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543615    2578 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543620    2578 flags.go:64] FLAG: --max-pods="110"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543623    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543626    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543629    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543632    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543635    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543638    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543641    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543649    2578 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 22:27:40.549159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543653    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543656    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543659    2578 flags.go:64] FLAG: --pod-cidr=""
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543662    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543668    2578 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543671    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543674    2578 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543677    2578 flags.go:64] FLAG: --port="10250"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543680    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543683    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b57b1ea67666b79b"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543686    2578 flags.go:64] FLAG: --qos-reserved=""
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543689    2578 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543692    2578 flags.go:64] FLAG: --register-node="true"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543695    2578 flags.go:64] FLAG: --register-with-taints=""
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543699    2578 flags.go:64] FLAG: --registry-burst="10"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543703    2578 flags.go:64] FLAG: --registry-qps="5"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543706    2578 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543709    2578 flags.go:64] FLAG: --reserved-memory=""
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543712    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543717    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543720    2578 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543723    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543726    2578 flags.go:64] FLAG: --runonce="false"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543729    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543732    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 22:27:40.549806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543735    2578 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543738    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543741    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543744    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543747    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543750    2578 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543753    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543756    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543759    2578 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543762    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543765    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543768    2578 flags.go:64] FLAG: --system-cgroups=""
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543771    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543776    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543779    2578 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543782    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543787    2578 flags.go:64] FLAG: --tls-min-version=""
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543790    2578 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543792    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543795    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543798    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543801    2578 flags.go:64] FLAG: --v="2"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543805    2578 flags.go:64] FLAG: --version="false"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543812    2578 flags.go:64] FLAG: --vmodule=""
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543816    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 22:27:40.550435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.543819    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543938    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543942    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543946    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543949    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543952    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543955    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543958    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543961    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543964    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543968    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543971    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543974    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543977    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543980    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543982    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543985    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543988    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543991    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:40.551105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543993    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543996    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.543999    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544002    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544004    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544007    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544009    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544012    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544014    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544017    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544020    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544027    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544030    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544033    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544037    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544040    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544043    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544045    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544048    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544051    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:40.551602 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544054    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544056    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544060    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544064    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544067    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544069    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544072    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544075    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544078    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544080    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544083    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544085    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544088    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544106    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544109    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544112    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544115    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544118    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544122    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:40.552118 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544125    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544129    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544134    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544136    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544140    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544143    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544147    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544151    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544154    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544157    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544159    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544162    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544165    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544168    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544170    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544173    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544176    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544179    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544182    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544184    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:40.552601 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544187    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544190    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544193    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544195    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544198    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544200    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544204    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544208    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544211    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544214    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544217    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544220    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544223    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544226    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544229    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544232    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544236    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544238    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544241    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:40.553105 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544253    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544256    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.544259    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.545186    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.552567    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.552588    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.553365    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.553377    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553458    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553464    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553467    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553470    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553473    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553476    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:40.553573 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553480    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553484    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553487    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553491    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553493    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553496    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553499    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553502    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553505    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553508    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553510    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553513    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553515    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553518    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553521    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553525    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553527    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553530    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553534    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553536    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:40.553948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553539    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553541    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553544    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553547    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553551    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553553    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553556    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553558    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553561    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553564    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553566    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553569    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553572    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553575    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553578    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553581    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553584    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553586    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553589    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553591    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:40.554540 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553594    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553597    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553600    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553602    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553605    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553607    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553611    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553614    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553616    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553619    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553622    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553640    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553645    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553649    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553652    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553656    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553659    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553662    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553665    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:40.555061 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553668    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553671    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553674    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553677    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553680    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553683    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553686    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553690    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553694    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553704    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553706    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553709    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553712    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553714    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553717    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553720    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553722    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553725    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553727    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553730    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:40.555628 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553733    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553736    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553739    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553741    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553744    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553753    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553755    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553758    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553761    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553764    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553767    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553769    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553772    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553774    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:40.556120 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.553780    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553926    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553931    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553934    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553937    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553940    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553943    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553946    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553949    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553952    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553955    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553958    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553961    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553963    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553966    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553969    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553971    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553974    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553977    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553980    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:40.556516 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553982    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553985    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553987    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553990    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.553998    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554001    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554004    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554007    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554009    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554012    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554014    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554017    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554019    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554022    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554025    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554027    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554030    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554033    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554035    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554037    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:40.556996 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554040    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554043    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554045    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554048    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554050    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554053    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554056    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554059    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554061    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554064    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554067    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554069    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554072    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554074    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554077    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554080    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554082    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554102    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554108    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554112    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:40.557513 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554115    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554119    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554123    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554126    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554129    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554131    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554134    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554136    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554139    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554142    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554146    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554150    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554153    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554156    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554158    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554161    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554164    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554167    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554170    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554172    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:40.558023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554175    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554178    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554181    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554184    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554186    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554189    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554191    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554195    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554199    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554202    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554211    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554214    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554218    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554221    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554224    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554227    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554230    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554233    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:40.558529 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554236    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:40.558958 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:40.554238    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:40.558958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.554244    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 22:27:40.558958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.554401    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 22:27:40.558958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.557278    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 22:27:40.558958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.558271    2578 server.go:1019] "Starting client certificate rotation"
Sep 15 22:27:40.558958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.558387    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 22:27:40.558958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.558428    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 22:27:40.561511 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.561490    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 22:27:40.566380 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.566340    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 22:27:40.577218 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.577196    2578 log.go:25] "Validated CRI v1 runtime API"
Sep 15 22:27:40.581457 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.581436    2578 log.go:25] "Validated CRI v1 image API"
Sep 15 22:27:40.582459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.582441    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 22:27:40.587397 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.587369    2578 fs.go:135] Filesystem UUIDs: map[0129dd98-a9ca-4319-aa01-ad0920717f9e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 80e9ccdc-ccb2-4ffa-b173-278403616ea3:/dev/nvme0n1p4]
Sep 15 22:27:40.587490 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.587395    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 22:27:40.590039 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.590018    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 22:27:40.592870 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.592736    2578 manager.go:216] Machine: {Timestamp:2026-09-15 22:27:40.591458436 +0000 UTC m=+0.345082553 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099450 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec232308612e066c472df05715a77416 SystemUUID:ec232308-612e-066c-472d-f05715a77416 BootID:d2419ce6-cf43-41ee-93bc-cb13a3fdafa0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fe:38:5b:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fe:38:5b:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:e0:3f:2c:cd:6b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 22:27:40.592870 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.592863    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 22:27:40.593004 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.592967    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 22:27:40.596365 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.596334    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 22:27:40.596519 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.596369    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-247.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 22:27:40.598005 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.597993    2578 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 22:27:40.598049 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.598007    2578 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 22:27:40.598049 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.598021    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 22:27:40.598634 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.598623    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 22:27:40.599417 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.599406    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 22:27:40.602820 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.602806    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 22:27:40.602965 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.602955    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 22:27:40.603009 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.602989    2578 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 22:27:40.603009 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.603005    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 22:27:40.603060 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.603022    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 22:27:40.603060 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.603032    2578 kubelet.go:398] "Adding apiserver pod source"
Sep 15 22:27:40.603060 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.603042    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 22:27:40.603521 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.603508    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 22:27:40.603573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.603528    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 22:27:40.606492 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.606468    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 22:27:40.608712 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.608695    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 22:27:40.608782 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.608728    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 22:27:40.610362 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610348    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 22:27:40.610420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610375    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 22:27:40.610420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610386    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 22:27:40.610420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610395    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 22:27:40.610420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610404    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 22:27:40.610420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610410    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 22:27:40.610549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610425    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 22:27:40.610549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610432    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 22:27:40.610549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610441    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 22:27:40.610549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610446    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 22:27:40.610549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610462    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 22:27:40.610549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.610467    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 22:27:40.611180 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.611170    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 22:27:40.611180 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.611181    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 22:27:40.615588 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.615562    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-247.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 22:27:40.615765 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.615639    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 22:27:40.615765 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.615684    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-247.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 22:27:40.617145 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.617126    2578 server.go:1268] "Started kubelet"
Sep 15 22:27:40.617277 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.617219    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 22:27:40.617335 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.617308    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 22:27:40.617335 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.617231    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 22:27:40.618165 ip-10-0-128-247 systemd[1]: Started Kubernetes Kubelet.
Sep 15 22:27:40.619484 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.619469    2578 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 22:27:40.621893 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.621875    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 22:27:40.622287 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.622264    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 22:27:40.623490 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.622526    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-247.ec2.internal.18d59f02a2a464f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-247.ec2.internal,UID:ip-10-0-128-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-247.ec2.internal,},FirstTimestamp:2026-09-15 22:27:40.617073912 +0000 UTC m=+0.370698032,LastTimestamp:2026-09-15 22:27:40.617073912 +0000 UTC m=+0.370698032,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-247.ec2.internal,}"
Sep 15 22:27:40.626217 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.626200    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 22:27:40.626217 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.626209    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 22:27:40.626339 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.626274    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:40.626900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.626792    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 22:27:40.626900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.626813    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 22:27:40.626900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.626797    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 22:27:40.626900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.626870    2578 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 22:27:40.626900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.626877    2578 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 22:27:40.627210 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.627193    2578 factory.go:55] Registering systemd factory
Sep 15 22:27:40.627257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.627215    2578 factory.go:223] Registration of the systemd container factory successfully
Sep 15 22:27:40.627450 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.627430    2578 factory.go:153] Registering CRI-O factory
Sep 15 22:27:40.627450 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.627449    2578 factory.go:223] Registration of the crio container factory successfully
Sep 15 22:27:40.627586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.627504    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 22:27:40.627586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.627527    2578 factory.go:103] Registering Raw factory
Sep 15 22:27:40.627586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.627545    2578 manager.go:1196] Started watching for new ooms in manager
Sep 15 22:27:40.628032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.628017    2578 manager.go:349] Starting recovery of all containers
Sep 15 22:27:40.637151 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.636901    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-247.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 15 22:27:40.637151 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.636973    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 15 22:27:40.641482 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.641463    2578 manager.go:354] Recovery completed
Sep 15 22:27:40.645377 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.645347    2578 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 15 22:27:40.649369 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.649354    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:40.651353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.651326    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:40.651446 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.651372    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:40.651446 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.651388    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:40.652076 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.652055    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 22:27:40.652076 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.652066    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:40.652076 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.652074    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 22:27:40.652292 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.652118    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 22:27:40.652859 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.652844    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:40.652934 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.652873    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:40.652934 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.652886    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:40.653400 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.653385    2578 policy_none.go:49] "None policy: Start"
Sep 15 22:27:40.653481 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.653404    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 22:27:40.653481 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.653417    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 22:27:40.654573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.654560    2578 policy_none.go:47] "Start"
Sep 15 22:27:40.662402 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.662315    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-247.ec2.internal.18d59f02a4af7307  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-247.ec2.internal,UID:ip-10-0-128-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-247.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-247.ec2.internal,},FirstTimestamp:2026-09-15 22:27:40.651352839 +0000 UTC m=+0.404976962,LastTimestamp:2026-09-15 22:27:40.651352839 +0000 UTC m=+0.404976962,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-247.ec2.internal,}"
Sep 15 22:27:40.669593 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.669492    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-247.ec2.internal.18d59f02a4afd86b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-247.ec2.internal,UID:ip-10-0-128-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-247.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-247.ec2.internal,},FirstTimestamp:2026-09-15 22:27:40.651378795 +0000 UTC m=+0.405002913,LastTimestamp:2026-09-15 22:27:40.651378795 +0000 UTC m=+0.405002913,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-247.ec2.internal,}"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.680039    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-247.ec2.internal.18d59f02a4b01136  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-247.ec2.internal,UID:ip-10-0-128-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-247.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-247.ec2.internal,},FirstTimestamp:2026-09-15 22:27:40.651393334 +0000 UTC m=+0.405017453,LastTimestamp:2026-09-15 22:27:40.651393334 +0000 UTC m=+0.405017453,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-247.ec2.internal,}"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.681084    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mx99q"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.683736    2578 manager.go:337] "Starting Device Plugin manager"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.683773    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.683787    2578 server.go:87] "Starting device plugin registration server"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.684201    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.684211    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.684343    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.684419    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.684426    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.686085    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.686179    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.688969    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mx99q"
Sep 15 22:27:40.693817 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.689428    2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-247.ec2.internal.18d59f02a4af7307\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-247.ec2.internal.18d59f02a4af7307  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-247.ec2.internal,UID:ip-10-0-128-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-247.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-247.ec2.internal,},FirstTimestamp:2026-09-15 22:27:40.651352839 +0000 UTC m=+0.404976962,LastTimestamp:2026-09-15 22:27:40.652858228 +0000 UTC m=+0.406482346,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-247.ec2.internal,}"
Sep 15 22:27:40.755053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.755002    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 22:27:40.756267 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.756247    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 22:27:40.756410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.756297    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 22:27:40.756410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.756319    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 22:27:40.756410 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.756357    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 22:27:40.758172 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.758142    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:40.785157 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.785040    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:40.786073 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.786053    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:40.786199 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.786088    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:40.786199 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.786120    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:40.786199 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.786158    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-247.ec2.internal"
Sep 15 22:27:40.793659 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.793639    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-247.ec2.internal"
Sep 15 22:27:40.793736 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.793664    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-247.ec2.internal\": node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:40.817205 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.817175    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:40.857179 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.857133    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal"]
Sep 15 22:27:40.857329 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.857223    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:40.858853 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.858834    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:40.858953 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.858866    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:40.858953 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.858876    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:40.860231 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.860215    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:40.860375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.860359    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:40.860452 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.860399    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:40.861389 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.861369    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:40.861499 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.861390    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:40.861499 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.861399    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:40.861499 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.861411    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:40.861499 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.861415    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:40.861499 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.861428    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:40.862505 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.862487    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:40.862592 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.862518    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:40.863351 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.863337    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:40.863427 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.863366    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:40.863427 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:40.863379    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:40.887036 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.887010    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-247.ec2.internal\" not found" node="ip-10-0-128-247.ec2.internal"
Sep 15 22:27:40.891835 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.891815    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-247.ec2.internal\" not found" node="ip-10-0-128-247.ec2.internal"
Sep 15 22:27:40.917698 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:40.917663    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:41.018295 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:41.018261    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:41.028605 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.028584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c28a4ec2fbe427bdfc7c14e0446ecb67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal\" (UID: \"c28a4ec2fbe427bdfc7c14e0446ecb67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.028703 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.028612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c28a4ec2fbe427bdfc7c14e0446ecb67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal\" (UID: \"c28a4ec2fbe427bdfc7c14e0446ecb67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.028703 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.028631    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/146a57ab6e613bbac55bc1f7a3764a51-config\") pod \"kube-apiserver-proxy-ip-10-0-128-247.ec2.internal\" (UID: \"146a57ab6e613bbac55bc1f7a3764a51\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.119009 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:41.118937    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:41.129293 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.129264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c28a4ec2fbe427bdfc7c14e0446ecb67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal\" (UID: \"c28a4ec2fbe427bdfc7c14e0446ecb67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.129348 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.129297    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c28a4ec2fbe427bdfc7c14e0446ecb67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal\" (UID: \"c28a4ec2fbe427bdfc7c14e0446ecb67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.129348 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.129312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/146a57ab6e613bbac55bc1f7a3764a51-config\") pod \"kube-apiserver-proxy-ip-10-0-128-247.ec2.internal\" (UID: \"146a57ab6e613bbac55bc1f7a3764a51\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.129414 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.129355    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c28a4ec2fbe427bdfc7c14e0446ecb67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal\" (UID: \"c28a4ec2fbe427bdfc7c14e0446ecb67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.129414 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.129364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c28a4ec2fbe427bdfc7c14e0446ecb67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal\" (UID: \"c28a4ec2fbe427bdfc7c14e0446ecb67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.129414 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.129411    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/146a57ab6e613bbac55bc1f7a3764a51-config\") pod \"kube-apiserver-proxy-ip-10-0-128-247.ec2.internal\" (UID: \"146a57ab6e613bbac55bc1f7a3764a51\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.189463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.189410    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.194206 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.194188    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.220086 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:41.220051    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:41.320632 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:41.320593    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:41.421248 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:41.421164    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:41.521719 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:41.521676    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:41.558251 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.558216    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 22:27:41.558696 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.558391    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 22:27:41.622595 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:41.622572    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:41.689462 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.689378    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:41.690795 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.690767    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:41.690911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.690800    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:41.690911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.690812    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:41.691888 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.691857    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 22:22:41 +0000 UTC" deadline="2028-04-14 01:09:00.145070095 +0000 UTC"
Sep 15 22:27:41.691946 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.691888    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13826h41m18.453184822s"
Sep 15 22:27:41.720122 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:41.720073    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod146a57ab6e613bbac55bc1f7a3764a51.slice/crio-b3739f8b91839836438663c431cfc839f07863f5770067d8b25667d40708316f WatchSource:0}: Error finding container b3739f8b91839836438663c431cfc839f07863f5770067d8b25667d40708316f: Status 404 returned error can't find the container with id b3739f8b91839836438663c431cfc839f07863f5770067d8b25667d40708316f
Sep 15 22:27:41.720366 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:41.720341    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc28a4ec2fbe427bdfc7c14e0446ecb67.slice/crio-788c086dc0449e9a415aece6890bf7b3b2a8690469a60a8c998ea402a951b54e WatchSource:0}: Error finding container 788c086dc0449e9a415aece6890bf7b3b2a8690469a60a8c998ea402a951b54e: Status 404 returned error can't find the container with id 788c086dc0449e9a415aece6890bf7b3b2a8690469a60a8c998ea402a951b54e
Sep 15 22:27:41.722902 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:41.722877    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-247.ec2.internal\" not found"
Sep 15 22:27:41.725490 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.725378    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:27:41.760157 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.760085    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal" event={"ID":"c28a4ec2fbe427bdfc7c14e0446ecb67","Type":"ContainerStarted","Data":"788c086dc0449e9a415aece6890bf7b3b2a8690469a60a8c998ea402a951b54e"}
Sep 15 22:27:41.761131 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.761088    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal" event={"ID":"146a57ab6e613bbac55bc1f7a3764a51","Type":"ContainerStarted","Data":"b3739f8b91839836438663c431cfc839f07863f5770067d8b25667d40708316f"}
Sep 15 22:27:41.774051 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.774022    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:41.826850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.826824    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.842053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.842021    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 22:27:41.844181 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.844161    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal"
Sep 15 22:27:41.860648 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.860623    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 22:27:41.994353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:41.994271    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:42.235654 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.235624    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:42.517329 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.517301    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:42.604249 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.604209    2578 apiserver.go:52] "Watching apiserver"
Sep 15 22:27:42.616036 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.616002    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 22:27:42.618306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.618269    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-77pzk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal","openshift-multus/multus-additional-cni-plugins-slcfb","openshift-multus/multus-zmr42","openshift-multus/network-metrics-daemon-tbgzs","openshift-network-diagnostics/network-check-target-rc44q","kube-system/konnectivity-agent-m4l8b","kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx","openshift-cluster-node-tuning-operator/tuned-xpwtg","openshift-network-operator/iptables-alerter-vz4vb","openshift-ovn-kubernetes/ovnkube-node-k6vk7","kube-system/global-pull-secret-syncer-dssx6","openshift-dns/node-resolver-gd8k6"]
Sep 15 22:27:42.619686 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.619663    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:27:42.620852 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.620824    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.622078 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.622056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.623617 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.623594    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 22:27:42.623854 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.623832    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 22:27:42.624124 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.624110    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7bpdd\""
Sep 15 22:27:42.624421 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.624402    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 22:27:42.624622 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.624606    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.624758 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.624740    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7m7v6\""
Sep 15 22:27:42.624997 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.624978    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 22:27:42.625644 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.625148    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 22:27:42.625644 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.625346    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:42.625644 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.625403    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:27:42.625847 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.625713    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hmjfv\""
Sep 15 22:27:42.629868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.625887    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 22:27:42.629868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.627021    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 22:27:42.629868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.627117    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:42.629868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.627197    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 22:27:42.629868 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.627200    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:27:42.629868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.628343    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 22:27:42.629868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.629490    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 22:27:42.629868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.629654    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vxqpm\""
Sep 15 22:27:42.630725 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.629930    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 22:27:42.630725 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.629936    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 22:27:42.630725 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.630173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.632113 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.631630    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.633411 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.633386    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 22:27:42.633517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.633412    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mzx62\""
Sep 15 22:27:42.633517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.633455    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 22:27:42.634002 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.633982    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.635869 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.635850    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.636536 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636417    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9712cd0c-ad1a-4b5a-aef5-693c83bd051d-host-slash\") pod \"iptables-alerter-vz4vb\" (UID: \"9712cd0c-ad1a-4b5a-aef5-693c83bd051d\") " pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.636536 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636451    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dddzn\" (UniqueName: \"kubernetes.io/projected/9712cd0c-ad1a-4b5a-aef5-693c83bd051d-kube-api-access-dddzn\") pod \"iptables-alerter-vz4vb\" (UID: \"9712cd0c-ad1a-4b5a-aef5-693c83bd051d\") " pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.636536 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636471    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-cnibin\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.636536 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636478    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pgtcl\""
Sep 15 22:27:42.636536 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636483    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 22:27:42.636536 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q44b\" (UniqueName: \"kubernetes.io/projected/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-kube-api-access-8q44b\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636541    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dca0482c-e29e-4e5b-b0e5-0e069d81fdba-hosts-file\") pod \"node-resolver-gd8k6\" (UID: \"dca0482c-e29e-4e5b-b0e5-0e069d81fdba\") " pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636609    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0d8ec9c1-59c0-459e-b19b-b86f00b0b55f-konnectivity-ca\") pod \"konnectivity-agent-m4l8b\" (UID: \"0d8ec9c1-59c0-459e-b19b-b86f00b0b55f\") " pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636639    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-system-cni-dir\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-cni-dir\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-cni-binary-copy\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636746    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-run-multus-certs\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636764    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbbzb\" (UniqueName: \"kubernetes.io/projected/7b02d944-b4ba-440f-8ec2-bec251299069-kube-api-access-dbbzb\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636783    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9712cd0c-ad1a-4b5a-aef5-693c83bd051d-iptables-alerter-script\") pod \"iptables-alerter-vz4vb\" (UID: \"9712cd0c-ad1a-4b5a-aef5-693c83bd051d\") " pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-var-lib-kubelet\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636855    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:42.636886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636881    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca0482c-e29e-4e5b-b0e5-0e069d81fdba-tmp-dir\") pod \"node-resolver-gd8k6\" (UID: \"dca0482c-e29e-4e5b-b0e5-0e069d81fdba\") " pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcc6k\" (UniqueName: \"kubernetes.io/projected/dca0482c-e29e-4e5b-b0e5-0e069d81fdba-kube-api-access-tcc6k\") pod \"node-resolver-gd8k6\" (UID: \"dca0482c-e29e-4e5b-b0e5-0e069d81fdba\") " pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636929    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c1381d9-2208-4034-89c3-458f4f74eb07-system-cni-dir\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636937    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636952    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c1381d9-2208-4034-89c3-458f4f74eb07-cni-binary-copy\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.636976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c1381d9-2208-4034-89c3-458f4f74eb07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637028    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637042    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-var-lib-cni-multus\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637087    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-daemon-config\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c1381d9-2208-4034-89c3-458f4f74eb07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637354    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c1381d9-2208-4034-89c3-458f4f74eb07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637360    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637381    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-run-netns\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-hostroot\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637425    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-etc-kubernetes\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637448    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8f78\" (UniqueName: \"kubernetes.io/projected/7c1381d9-2208-4034-89c3-458f4f74eb07-kube-api-access-h8f78\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c1381d9-2208-4034-89c3-458f4f74eb07-cnibin\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637504    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:42.638187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637548    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637426    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637542    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-socket-dir-parent\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637697    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-conf-dir\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0d8ec9c1-59c0-459e-b19b-b86f00b0b55f-agent-certs\") pod \"konnectivity-agent-m4l8b\" (UID: \"0d8ec9c1-59c0-459e-b19b-b86f00b0b55f\") " pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.637716    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637734    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-run-k8s-cni-cncf-io\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.637756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-var-lib-cni-bin\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.638019    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.638146    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.638153    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.638588    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xqbx5\""
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.638609    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ldz54\""
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.638825    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 22:27:42.639032 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.638862    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 22:27:42.639746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.639539    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.642183 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.642163    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 22:27:42.642332 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.642300    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7ssbg\""
Sep 15 22:27:42.642422 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.642397    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 22:27:42.642482 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.642402    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 22:27:42.643420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.643397    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 22:27:42.667190 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.667162    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zt9vh"
Sep 15 22:27:42.674987 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.674954    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zt9vh"
Sep 15 22:27:42.728578 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.728547    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 22:27:42.738447 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.738415    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.738447 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.738456    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b32c671-b247-4256-878e-a286c276b9ca-dbus\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:42.738697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.738486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0d8ec9c1-59c0-459e-b19b-b86f00b0b55f-agent-certs\") pod \"konnectivity-agent-m4l8b\" (UID: \"0d8ec9c1-59c0-459e-b19b-b86f00b0b55f\") " pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:27:42.738697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.738512    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-run-k8s-cni-cncf-io\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.738697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.738571    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-run-k8s-cni-cncf-io\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.738697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.738573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-var-lib-cni-bin\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.738697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.738610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-kubelet\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.738697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.738635    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-run-ovn\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.738697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.738646    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-var-lib-cni-bin\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.739075 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-log-socket\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.739163 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-cni-bin\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.739163 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739149    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/59d2402a-a6ba-4c47-bbbd-7d70390292a9-env-overrides\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.739304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739180    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-sysconfig\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.739373 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-run-multus-certs\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.739430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbbzb\" (UniqueName: \"kubernetes.io/projected/7b02d944-b4ba-440f-8ec2-bec251299069-kube-api-access-dbbzb\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:42.739430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-run-multus-certs\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.739552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkzmf\" (UniqueName: \"kubernetes.io/projected/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-kube-api-access-nkzmf\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.739552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-systemd-units\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.739552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9712cd0c-ad1a-4b5a-aef5-693c83bd051d-iptables-alerter-script\") pod \"iptables-alerter-vz4vb\" (UID: \"9712cd0c-ad1a-4b5a-aef5-693c83bd051d\") " pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.739552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:42.739744 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c1381d9-2208-4034-89c3-458f4f74eb07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.739744 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739608    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-run\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.739744 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739641    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-socket-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.739744 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-etc-openvswitch\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.739744 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739697    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/59d2402a-a6ba-4c47-bbbd-7d70390292a9-ovn-node-metrics-cert\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.739744 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-daemon-config\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.740005 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739754    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 22:27:42.740005 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739769    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c1381d9-2208-4034-89c3-458f4f74eb07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.740005 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739945    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.740005 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.739978    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-sys-fs\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.740206 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740005    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmvn9\" (UniqueName: \"kubernetes.io/projected/c592487e-05b6-445b-ba50-08f1899afeaa-kube-api-access-cmvn9\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.740206 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740038    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-run-netns\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.740206 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-etc-kubernetes\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.740206 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8f78\" (UniqueName: \"kubernetes.io/projected/7c1381d9-2208-4034-89c3-458f4f74eb07-kube-api-access-h8f78\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.740403 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740233    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/59d2402a-a6ba-4c47-bbbd-7d70390292a9-ovnkube-script-lib\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.740403 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c1381d9-2208-4034-89c3-458f4f74eb07-cnibin\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.740403 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:42.740586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-host\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.740586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740433    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-tmp\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.740586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:42.740586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-socket-dir-parent\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.740769 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-conf-dir\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.740769 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740639    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-sysctl-conf\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.740769 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740673    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d191e41f-4cad-467f-9bdc-86a9b2d61d19-serviceca\") pod \"node-ca-77pzk\" (UID: \"d191e41f-4cad-467f-9bdc-86a9b2d61d19\") " pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.740900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.740900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/59d2402a-a6ba-4c47-bbbd-7d70390292a9-ovnkube-config\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.740995 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-device-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.740995 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-etc-kubernetes\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.740995 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9712cd0c-ad1a-4b5a-aef5-693c83bd051d-host-slash\") pod \"iptables-alerter-vz4vb\" (UID: \"9712cd0c-ad1a-4b5a-aef5-693c83bd051d\") " pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.740995 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.740973    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dddzn\" (UniqueName: \"kubernetes.io/projected/9712cd0c-ad1a-4b5a-aef5-693c83bd051d-kube-api-access-dddzn\") pod \"iptables-alerter-vz4vb\" (UID: \"9712cd0c-ad1a-4b5a-aef5-693c83bd051d\") " pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.741208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-socket-dir-parent\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.741208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-cnibin\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.741208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741050    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q44b\" (UniqueName: \"kubernetes.io/projected/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-kube-api-access-8q44b\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.741208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dca0482c-e29e-4e5b-b0e5-0e069d81fdba-hosts-file\") pod \"node-resolver-gd8k6\" (UID: \"dca0482c-e29e-4e5b-b0e5-0e069d81fdba\") " pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.741208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-modprobe-d\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.741208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-daemon-config\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.741208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741155    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c1381d9-2208-4034-89c3-458f4f74eb07-cnibin\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.741526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741264    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-kubernetes\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.741526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741300    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-registration-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.741526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0d8ec9c1-59c0-459e-b19b-b86f00b0b55f-konnectivity-ca\") pod \"konnectivity-agent-m4l8b\" (UID: \"0d8ec9c1-59c0-459e-b19b-b86f00b0b55f\") " pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:27:42.741526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-system-cni-dir\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.741526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c1381d9-2208-4034-89c3-458f4f74eb07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.741526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-cni-dir\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.741526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-cni-binary-copy\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.741840 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741653    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-cni-dir\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.741840 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741726    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-sysctl-d\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.741840 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.741748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-cnibin\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.741840 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.741823    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:42.742135 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9712cd0c-ad1a-4b5a-aef5-693c83bd051d-iptables-alerter-script\") pod \"iptables-alerter-vz4vb\" (UID: \"9712cd0c-ad1a-4b5a-aef5-693c83bd051d\") " pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.742212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-var-lib-openvswitch\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.742212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-cni-netd\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.742320 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9712cd0c-ad1a-4b5a-aef5-693c83bd051d-host-slash\") pod \"iptables-alerter-vz4vb\" (UID: \"9712cd0c-ad1a-4b5a-aef5-693c83bd051d\") " pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.742320 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742303    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-system-cni-dir\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.742407 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742330    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-multus-conf-dir\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.742551 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.742533    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs podName:7b02d944-b4ba-440f-8ec2-bec251299069 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:43.242507326 +0000 UTC m=+2.996131450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs") pod "network-metrics-daemon-tbgzs" (UID: "7b02d944-b4ba-440f-8ec2-bec251299069") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:42.742664 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742649    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-cni-binary-copy\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.742718 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c1381d9-2208-4034-89c3-458f4f74eb07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.742890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-var-lib-kubelet\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.742941 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dca0482c-e29e-4e5b-b0e5-0e069d81fdba-hosts-file\") pod \"node-resolver-gd8k6\" (UID: \"dca0482c-e29e-4e5b-b0e5-0e069d81fdba\") " pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.742941 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.742911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca0482c-e29e-4e5b-b0e5-0e069d81fdba-tmp-dir\") pod \"node-resolver-gd8k6\" (UID: \"dca0482c-e29e-4e5b-b0e5-0e069d81fdba\") " pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.743059 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0d8ec9c1-59c0-459e-b19b-b86f00b0b55f-konnectivity-ca\") pod \"konnectivity-agent-m4l8b\" (UID: \"0d8ec9c1-59c0-459e-b19b-b86f00b0b55f\") " pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:27:42.743135 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743044    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcc6k\" (UniqueName: \"kubernetes.io/projected/dca0482c-e29e-4e5b-b0e5-0e069d81fdba-kube-api-access-tcc6k\") pod \"node-resolver-gd8k6\" (UID: \"dca0482c-e29e-4e5b-b0e5-0e069d81fdba\") " pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.743135 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743129    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c1381d9-2208-4034-89c3-458f4f74eb07-system-cni-dir\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.743228 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c1381d9-2208-4034-89c3-458f4f74eb07-system-cni-dir\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.743269 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c1381d9-2208-4034-89c3-458f4f74eb07-cni-binary-copy\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.743321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-var-lib-kubelet\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.743371 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c1381d9-2208-4034-89c3-458f4f74eb07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.743421 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-systemd\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.743421 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743409    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca0482c-e29e-4e5b-b0e5-0e069d81fdba-tmp-dir\") pod \"node-resolver-gd8k6\" (UID: \"dca0482c-e29e-4e5b-b0e5-0e069d81fdba\") " pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.743517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-tuned\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.743517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-var-lib-cni-multus\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.743621 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743566    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-lib-modules\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.743621 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743593    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.743720 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743647    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-run-systemd\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.743720 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skjnk\" (UniqueName: \"kubernetes.io/projected/59d2402a-a6ba-4c47-bbbd-7d70390292a9-kube-api-access-skjnk\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.743720 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d191e41f-4cad-467f-9bdc-86a9b2d61d19-host\") pod \"node-ca-77pzk\" (UID: \"d191e41f-4cad-467f-9bdc-86a9b2d61d19\") " pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.743865 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b32c671-b247-4256-878e-a286c276b9ca-kubelet-config\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:42.743865 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743790    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-run-netns\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.743865 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743821    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-hostroot\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.743865 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-sys\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.744057 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743896    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-slash\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.744057 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743953    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-run-openvswitch\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.744057 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743981    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-node-log\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.744057 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.744015    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzsdx\" (UniqueName: \"kubernetes.io/projected/d191e41f-4cad-467f-9bdc-86a9b2d61d19-kube-api-access-nzsdx\") pod \"node-ca-77pzk\" (UID: \"d191e41f-4cad-467f-9bdc-86a9b2d61d19\") " pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.744057 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.744047    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-var-lib-kubelet\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.744306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.744178    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-var-lib-cni-multus\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.744360 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.744341    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0d8ec9c1-59c0-459e-b19b-b86f00b0b55f-agent-certs\") pod \"konnectivity-agent-m4l8b\" (UID: \"0d8ec9c1-59c0-459e-b19b-b86f00b0b55f\") " pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:27:42.744452 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.744435    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-hostroot\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.744503 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.743788    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c1381d9-2208-4034-89c3-458f4f74eb07-cni-binary-copy\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.744503 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.744436    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-host-run-netns\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.744823 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.744769    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c1381d9-2208-4034-89c3-458f4f74eb07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.746935 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.746913    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:42.747044 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.746942    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:42.747044 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.746953    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8z2t8 for pod openshift-network-diagnostics/network-check-target-rc44q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:42.747044 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.747010    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8 podName:bae50b7e-fb4b-4da6-b71e-c824b2b6d295 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:43.24699725 +0000 UTC m=+3.000621370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8z2t8" (UniqueName: "kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8") pod "network-check-target-rc44q" (UID: "bae50b7e-fb4b-4da6-b71e-c824b2b6d295") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:42.748176 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.748151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbbzb\" (UniqueName: \"kubernetes.io/projected/7b02d944-b4ba-440f-8ec2-bec251299069-kube-api-access-dbbzb\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:42.750975 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.750870    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8f78\" (UniqueName: \"kubernetes.io/projected/7c1381d9-2208-4034-89c3-458f4f74eb07-kube-api-access-h8f78\") pod \"multus-additional-cni-plugins-slcfb\" (UID: \"7c1381d9-2208-4034-89c3-458f4f74eb07\") " pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.752899 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.752874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dddzn\" (UniqueName: \"kubernetes.io/projected/9712cd0c-ad1a-4b5a-aef5-693c83bd051d-kube-api-access-dddzn\") pod \"iptables-alerter-vz4vb\" (UID: \"9712cd0c-ad1a-4b5a-aef5-693c83bd051d\") " pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.753122 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.753082    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcc6k\" (UniqueName: \"kubernetes.io/projected/dca0482c-e29e-4e5b-b0e5-0e069d81fdba-kube-api-access-tcc6k\") pod \"node-resolver-gd8k6\" (UID: \"dca0482c-e29e-4e5b-b0e5-0e069d81fdba\") " pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.753555 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.753532    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q44b\" (UniqueName: \"kubernetes.io/projected/002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc-kube-api-access-8q44b\") pod \"multus-zmr42\" (UID: \"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc\") " pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.844994 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.844909    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-run-openvswitch\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.844994 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.844949    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-node-log\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.844994 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.844975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzsdx\" (UniqueName: \"kubernetes.io/projected/d191e41f-4cad-467f-9bdc-86a9b2d61d19-kube-api-access-nzsdx\") pod \"node-ca-77pzk\" (UID: \"d191e41f-4cad-467f-9bdc-86a9b2d61d19\") " pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.844997    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-run-openvswitch\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.844999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-var-lib-kubelet\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-node-log\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845050    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-var-lib-kubelet\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b32c671-b247-4256-878e-a286c276b9ca-dbus\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-kubelet\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845180    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b32c671-b247-4256-878e-a286c276b9ca-dbus\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-run-ovn\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-log-socket\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845227    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-kubelet\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-cni-bin\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-run-ovn\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/59d2402a-a6ba-4c47-bbbd-7d70390292a9-env-overrides\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-log-socket\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-cni-bin\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.845282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-sysconfig\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845324    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-sysconfig\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkzmf\" (UniqueName: \"kubernetes.io/projected/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-kube-api-access-nkzmf\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-systemd-units\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-run\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845421    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-socket-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-etc-openvswitch\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/59d2402a-a6ba-4c47-bbbd-7d70390292a9-ovn-node-metrics-cert\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-run\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-systemd-units\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-sys-fs\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845536    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-etc-openvswitch\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845549    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmvn9\" (UniqueName: \"kubernetes.io/projected/c592487e-05b6-445b-ba50-08f1899afeaa-kube-api-access-cmvn9\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-run-netns\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845592    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-sys-fs\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845597    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-socket-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.846053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/59d2402a-a6ba-4c47-bbbd-7d70390292a9-ovnkube-script-lib\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845651    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-run-netns\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-host\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845686    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-tmp\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-sysctl-conf\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845759    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/59d2402a-a6ba-4c47-bbbd-7d70390292a9-env-overrides\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d191e41f-4cad-467f-9bdc-86a9b2d61d19-serviceca\") pod \"node-ca-77pzk\" (UID: \"d191e41f-4cad-467f-9bdc-86a9b2d61d19\") " pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845811    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/59d2402a-a6ba-4c47-bbbd-7d70390292a9-ovnkube-config\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.845858    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-device-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:42.845911    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret podName:0b32c671-b247-4256-878e-a286c276b9ca nodeName:}" failed. No retries permitted until 2026-09-15 22:27:43.345892839 +0000 UTC m=+3.099516959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret") pod "global-pull-secret-syncer-dssx6" (UID: "0b32c671-b247-4256-878e-a286c276b9ca") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845737    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-host\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-sysctl-conf\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845938    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.846850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-device-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.845981    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-modprobe-d\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-kubernetes\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-registration-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-sysctl-d\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-var-lib-openvswitch\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-registration-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846126    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-modprobe-d\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-cni-netd\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-systemd\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-var-lib-openvswitch\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-tuned\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-lib-modules\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846264    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-sysctl-d\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846330    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-run-systemd\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skjnk\" (UniqueName: \"kubernetes.io/projected/59d2402a-a6ba-4c47-bbbd-7d70390292a9-kube-api-access-skjnk\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.847735 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846374    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-lib-modules\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846381    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d191e41f-4cad-467f-9bdc-86a9b2d61d19-host\") pod \"node-ca-77pzk\" (UID: \"d191e41f-4cad-467f-9bdc-86a9b2d61d19\") " pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b32c671-b247-4256-878e-a286c276b9ca-kubelet-config\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-cni-netd\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846128    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-kubernetes\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-sys\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/59d2402a-a6ba-4c47-bbbd-7d70390292a9-ovnkube-script-lib\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846463    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-slash\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846512    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c592487e-05b6-445b-ba50-08f1899afeaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d191e41f-4cad-467f-9bdc-86a9b2d61d19-serviceca\") pod \"node-ca-77pzk\" (UID: \"d191e41f-4cad-467f-9bdc-86a9b2d61d19\") " pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846531    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-host-slash\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b32c671-b247-4256-878e-a286c276b9ca-kubelet-config\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-sys\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846583    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/59d2402a-a6ba-4c47-bbbd-7d70390292a9-run-systemd\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d191e41f-4cad-467f-9bdc-86a9b2d61d19-host\") pod \"node-ca-77pzk\" (UID: \"d191e41f-4cad-467f-9bdc-86a9b2d61d19\") " pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846620    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-systemd\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.848544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.846643    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/59d2402a-a6ba-4c47-bbbd-7d70390292a9-ovnkube-config\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.849057 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.848570    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-etc-tuned\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.849057 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.848610    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-tmp\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.849057 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.848725    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/59d2402a-a6ba-4c47-bbbd-7d70390292a9-ovn-node-metrics-cert\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.859396 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.859351    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkzmf\" (UniqueName: \"kubernetes.io/projected/11f3daa5-e359-45dd-8ed1-38ec9eb21ec8-kube-api-access-nkzmf\") pod \"tuned-xpwtg\" (UID: \"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8\") " pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.860076 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.860045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skjnk\" (UniqueName: \"kubernetes.io/projected/59d2402a-a6ba-4c47-bbbd-7d70390292a9-kube-api-access-skjnk\") pod \"ovnkube-node-k6vk7\" (UID: \"59d2402a-a6ba-4c47-bbbd-7d70390292a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.860250 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.860227    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzsdx\" (UniqueName: \"kubernetes.io/projected/d191e41f-4cad-467f-9bdc-86a9b2d61d19-kube-api-access-nzsdx\") pod \"node-ca-77pzk\" (UID: \"d191e41f-4cad-467f-9bdc-86a9b2d61d19\") " pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.860315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.860297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmvn9\" (UniqueName: \"kubernetes.io/projected/c592487e-05b6-445b-ba50-08f1899afeaa-kube-api-access-cmvn9\") pod \"aws-ebs-csi-driver-node-2bglx\" (UID: \"c592487e-05b6-445b-ba50-08f1899afeaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:42.934603 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.934559    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:27:42.945660 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.945408    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vz4vb"
Sep 15 22:27:42.954727 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.954693    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-slcfb"
Sep 15 22:27:42.961833 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.961794    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zmr42"
Sep 15 22:27:42.968919 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.968890    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gd8k6"
Sep 15 22:27:42.977881 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.977852    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:27:42.984013 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.983979    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-77pzk"
Sep 15 22:27:42.992183 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.992134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xpwtg"
Sep 15 22:27:42.997109 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:42.997081    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx"
Sep 15 22:27:43.248984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.248946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:43.249188 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.249001    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:43.249188 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:43.249136    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:43.249188 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:43.249156    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:43.249188 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:43.249179    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:43.249356 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:43.249193    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8z2t8 for pod openshift-network-diagnostics/network-check-target-rc44q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:43.249356 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:43.249209    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs podName:7b02d944-b4ba-440f-8ec2-bec251299069 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:44.249189523 +0000 UTC m=+4.002813634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs") pod "network-metrics-daemon-tbgzs" (UID: "7b02d944-b4ba-440f-8ec2-bec251299069") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:43.249356 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:43.249245    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8 podName:bae50b7e-fb4b-4da6-b71e-c824b2b6d295 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:44.249228071 +0000 UTC m=+4.002852177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8z2t8" (UniqueName: "kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8") pod "network-check-target-rc44q" (UID: "bae50b7e-fb4b-4da6-b71e-c824b2b6d295") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:43.350171 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.350095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:43.350354 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:43.350232    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:43.350354 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:43.350314    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret podName:0b32c671-b247-4256-878e-a286c276b9ca nodeName:}" failed. No retries permitted until 2026-09-15 22:27:44.350290606 +0000 UTC m=+4.103914719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret") pod "global-pull-secret-syncer-dssx6" (UID: "0b32c671-b247-4256-878e-a286c276b9ca") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:43.678803 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.678751    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 22:22:43 +0000 UTC" deadline="2028-04-09 13:45:47.295948768 +0000 UTC"
Sep 15 22:27:43.678803 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.678795    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13719h18m3.61715779s"
Sep 15 22:27:43.685823 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:43.685792    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod002b8e1b_4e2e_44cd_b110_24f8ff8cd2dc.slice/crio-632004e1fec7c27dfbe7441206e8f88342193ee925587d836d4d9ce1af881361 WatchSource:0}: Error finding container 632004e1fec7c27dfbe7441206e8f88342193ee925587d836d4d9ce1af881361: Status 404 returned error can't find the container with id 632004e1fec7c27dfbe7441206e8f88342193ee925587d836d4d9ce1af881361
Sep 15 22:27:43.686961 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:43.686936    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d2402a_a6ba_4c47_bbbd_7d70390292a9.slice/crio-31db8a54298a982177d9aeae17a9bc28e9b5464d665692e26bfec3975ea364e7 WatchSource:0}: Error finding container 31db8a54298a982177d9aeae17a9bc28e9b5464d665692e26bfec3975ea364e7: Status 404 returned error can't find the container with id 31db8a54298a982177d9aeae17a9bc28e9b5464d665692e26bfec3975ea364e7
Sep 15 22:27:43.688736 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:43.688609    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c1381d9_2208_4034_89c3_458f4f74eb07.slice/crio-35fdbda5bb09dadf0ef2ff5f3be21934a8be653b60218ec7887c82710b89ca4c WatchSource:0}: Error finding container 35fdbda5bb09dadf0ef2ff5f3be21934a8be653b60218ec7887c82710b89ca4c: Status 404 returned error can't find the container with id 35fdbda5bb09dadf0ef2ff5f3be21934a8be653b60218ec7887c82710b89ca4c
Sep 15 22:27:43.692092 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:43.692066    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca0482c_e29e_4e5b_b0e5_0e069d81fdba.slice/crio-e342376266d65d5042bb37e269f855e57156d99cfd3efff98538eba7522b558d WatchSource:0}: Error finding container e342376266d65d5042bb37e269f855e57156d99cfd3efff98538eba7522b558d: Status 404 returned error can't find the container with id e342376266d65d5042bb37e269f855e57156d99cfd3efff98538eba7522b558d
Sep 15 22:27:43.692959 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:43.692899    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc592487e_05b6_445b_ba50_08f1899afeaa.slice/crio-325ed5461fe8ea12fe6e99c50a1333e27771856db97e2e1bda5c499706a84888 WatchSource:0}: Error finding container 325ed5461fe8ea12fe6e99c50a1333e27771856db97e2e1bda5c499706a84888: Status 404 returned error can't find the container with id 325ed5461fe8ea12fe6e99c50a1333e27771856db97e2e1bda5c499706a84888
Sep 15 22:27:43.694057 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:43.693958    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9712cd0c_ad1a_4b5a_aef5_693c83bd051d.slice/crio-736df7ab8d76c2fae6a9d26a2fdedaec8247b2ba1bc8729ac988d102e157d541 WatchSource:0}: Error finding container 736df7ab8d76c2fae6a9d26a2fdedaec8247b2ba1bc8729ac988d102e157d541: Status 404 returned error can't find the container with id 736df7ab8d76c2fae6a9d26a2fdedaec8247b2ba1bc8729ac988d102e157d541
Sep 15 22:27:43.695829 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:43.695807    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f3daa5_e359_45dd_8ed1_38ec9eb21ec8.slice/crio-d40c5b832b504ef989dd2c08e8be332698e3c1a87e1bfa97ba64cb0c411f4369 WatchSource:0}: Error finding container d40c5b832b504ef989dd2c08e8be332698e3c1a87e1bfa97ba64cb0c411f4369: Status 404 returned error can't find the container with id d40c5b832b504ef989dd2c08e8be332698e3c1a87e1bfa97ba64cb0c411f4369
Sep 15 22:27:43.696835 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:43.696768    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd191e41f_4cad_467f_9bdc_86a9b2d61d19.slice/crio-b22fd8609d18ac32416453b4d50f066c2e766126ffb837db72f345bfb53357ad WatchSource:0}: Error finding container b22fd8609d18ac32416453b4d50f066c2e766126ffb837db72f345bfb53357ad: Status 404 returned error can't find the container with id b22fd8609d18ac32416453b4d50f066c2e766126ffb837db72f345bfb53357ad
Sep 15 22:27:43.699454 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:27:43.699417    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d8ec9c1_59c0_459e_b19b_b86f00b0b55f.slice/crio-cc0f18a1bdb73df122ae986ee23b8ed241a80f663f74d5ba926fe59855a6eb82 WatchSource:0}: Error finding container cc0f18a1bdb73df122ae986ee23b8ed241a80f663f74d5ba926fe59855a6eb82: Status 404 returned error can't find the container with id cc0f18a1bdb73df122ae986ee23b8ed241a80f663f74d5ba926fe59855a6eb82
Sep 15 22:27:43.756603 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.756569    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:43.756732 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:43.756685    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:27:43.765920 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.765878    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m4l8b" event={"ID":"0d8ec9c1-59c0-459e-b19b-b86f00b0b55f","Type":"ContainerStarted","Data":"cc0f18a1bdb73df122ae986ee23b8ed241a80f663f74d5ba926fe59855a6eb82"}
Sep 15 22:27:43.767284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.767245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xpwtg" event={"ID":"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8","Type":"ContainerStarted","Data":"d40c5b832b504ef989dd2c08e8be332698e3c1a87e1bfa97ba64cb0c411f4369"}
Sep 15 22:27:43.768548 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.768522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vz4vb" event={"ID":"9712cd0c-ad1a-4b5a-aef5-693c83bd051d","Type":"ContainerStarted","Data":"736df7ab8d76c2fae6a9d26a2fdedaec8247b2ba1bc8729ac988d102e157d541"}
Sep 15 22:27:43.769867 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.769840    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx" event={"ID":"c592487e-05b6-445b-ba50-08f1899afeaa","Type":"ContainerStarted","Data":"325ed5461fe8ea12fe6e99c50a1333e27771856db97e2e1bda5c499706a84888"}
Sep 15 22:27:43.771464 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.771132    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gd8k6" event={"ID":"dca0482c-e29e-4e5b-b0e5-0e069d81fdba","Type":"ContainerStarted","Data":"e342376266d65d5042bb37e269f855e57156d99cfd3efff98538eba7522b558d"}
Sep 15 22:27:43.772466 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.772437    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slcfb" event={"ID":"7c1381d9-2208-4034-89c3-458f4f74eb07","Type":"ContainerStarted","Data":"35fdbda5bb09dadf0ef2ff5f3be21934a8be653b60218ec7887c82710b89ca4c"}
Sep 15 22:27:43.773856 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.773831    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zmr42" event={"ID":"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc","Type":"ContainerStarted","Data":"632004e1fec7c27dfbe7441206e8f88342193ee925587d836d4d9ce1af881361"}
Sep 15 22:27:43.775112 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.775077    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-77pzk" event={"ID":"d191e41f-4cad-467f-9bdc-86a9b2d61d19","Type":"ContainerStarted","Data":"b22fd8609d18ac32416453b4d50f066c2e766126ffb837db72f345bfb53357ad"}
Sep 15 22:27:43.776503 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:43.776480    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" event={"ID":"59d2402a-a6ba-4c47-bbbd-7d70390292a9","Type":"ContainerStarted","Data":"31db8a54298a982177d9aeae17a9bc28e9b5464d665692e26bfec3975ea364e7"}
Sep 15 22:27:44.256593 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.256547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:44.256815 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.256622    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:44.256886 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.256837    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:44.256886 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.256856    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:44.256886 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.256870    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8z2t8 for pod openshift-network-diagnostics/network-check-target-rc44q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:44.257044 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.256934    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8 podName:bae50b7e-fb4b-4da6-b71e-c824b2b6d295 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:46.256914367 +0000 UTC m=+6.010538479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8z2t8" (UniqueName: "kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8") pod "network-check-target-rc44q" (UID: "bae50b7e-fb4b-4da6-b71e-c824b2b6d295") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:44.257423 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.257399    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:44.257505 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.257462    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs podName:7b02d944-b4ba-440f-8ec2-bec251299069 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:46.257444575 +0000 UTC m=+6.011068697 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs") pod "network-metrics-daemon-tbgzs" (UID: "7b02d944-b4ba-440f-8ec2-bec251299069") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:44.357889 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.357841    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:44.358097 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.357992    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:44.358097 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.358067    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret podName:0b32c671-b247-4256-878e-a286c276b9ca nodeName:}" failed. No retries permitted until 2026-09-15 22:27:46.358046296 +0000 UTC m=+6.111670424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret") pod "global-pull-secret-syncer-dssx6" (UID: "0b32c671-b247-4256-878e-a286c276b9ca") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:44.679958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.679828    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 22:22:43 +0000 UTC" deadline="2028-04-19 21:40:52.377940177 +0000 UTC"
Sep 15 22:27:44.679958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.679900    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13967h13m7.698045214s"
Sep 15 22:27:44.758847 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.758287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:44.758847 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.758444    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:27:44.759409 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.759213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:44.759409 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:44.759335    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:27:44.788624 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.788149    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal" event={"ID":"146a57ab6e613bbac55bc1f7a3764a51","Type":"ContainerStarted","Data":"92314c40e6d50142a756fb63e0937cd3e40d5c3d4782cc5c5b44ac92a13544aa"}
Sep 15 22:27:44.797768 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.797726    2578 generic.go:356] "Generic (PLEG): container finished" podID="c28a4ec2fbe427bdfc7c14e0446ecb67" containerID="1fe43fbf939e576d761f1fd953e03ed521fa3796cd3686bda12775edc005f6f6" exitCode=0
Sep 15 22:27:44.797980 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.797784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal" event={"ID":"c28a4ec2fbe427bdfc7c14e0446ecb67","Type":"ContainerDied","Data":"1fe43fbf939e576d761f1fd953e03ed521fa3796cd3686bda12775edc005f6f6"}
Sep 15 22:27:44.804884 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:44.804622    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-247.ec2.internal" podStartSLOduration=2.804600689 podStartE2EDuration="2.804600689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:44.804399573 +0000 UTC m=+4.558023703" watchObservedRunningTime="2026-09-15 22:27:44.804600689 +0000 UTC m=+4.558224823"
Sep 15 22:27:45.756997 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:45.756953    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:45.757559 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:45.757093    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:27:45.823673 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:45.823574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal" event={"ID":"c28a4ec2fbe427bdfc7c14e0446ecb67","Type":"ContainerStarted","Data":"f773fdd255251322f96f0756fd671d2042d15e21e477a01b9fa34e8c95d126fd"}
Sep 15 22:27:45.853016 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:45.852569    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-247.ec2.internal" podStartSLOduration=3.852537233 podStartE2EDuration="3.852537233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:45.851588599 +0000 UTC m=+5.605212730" watchObservedRunningTime="2026-09-15 22:27:45.852537233 +0000 UTC m=+5.606161365"
Sep 15 22:27:46.273235 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:46.273189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:46.273470 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:46.273288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:46.273470 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.273412    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:46.273470 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.273441    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:46.273470 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.273471    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:46.273687 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.273485    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs podName:7b02d944-b4ba-440f-8ec2-bec251299069 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:50.27346241 +0000 UTC m=+10.027086518 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs") pod "network-metrics-daemon-tbgzs" (UID: "7b02d944-b4ba-440f-8ec2-bec251299069") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:46.273687 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.273486    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8z2t8 for pod openshift-network-diagnostics/network-check-target-rc44q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:46.273687 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.273533    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8 podName:bae50b7e-fb4b-4da6-b71e-c824b2b6d295 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:50.273522006 +0000 UTC m=+10.027146116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8z2t8" (UniqueName: "kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8") pod "network-check-target-rc44q" (UID: "bae50b7e-fb4b-4da6-b71e-c824b2b6d295") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:46.374005 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:46.373961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:46.374231 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.374124    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:46.374307 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.374255    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret podName:0b32c671-b247-4256-878e-a286c276b9ca nodeName:}" failed. No retries permitted until 2026-09-15 22:27:50.37423021 +0000 UTC m=+10.127854322 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret") pod "global-pull-secret-syncer-dssx6" (UID: "0b32c671-b247-4256-878e-a286c276b9ca") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:46.757779 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:46.757245    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:46.757779 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:46.757278    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:46.757779 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.757402    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:27:46.757779 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:46.757507    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:27:47.757362 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:47.757318    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:47.757592 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:47.757474    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:27:48.757201 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:48.757096    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:48.757694 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:48.757261    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:27:48.757694 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:48.757111    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:48.757789 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:48.757745    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:27:49.757741 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:49.757454    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:49.758285 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:49.757864    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:27:50.313935 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:50.313432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:50.313935 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:50.313500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:50.313935 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.313695    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:50.313935 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.313720    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:50.313935 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.313735    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8z2t8 for pod openshift-network-diagnostics/network-check-target-rc44q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:50.313935 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.313797    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8 podName:bae50b7e-fb4b-4da6-b71e-c824b2b6d295 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:58.31377703 +0000 UTC m=+18.067401142 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8z2t8" (UniqueName: "kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8") pod "network-check-target-rc44q" (UID: "bae50b7e-fb4b-4da6-b71e-c824b2b6d295") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:50.313935 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.313873    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:50.313935 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.313907    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs podName:7b02d944-b4ba-440f-8ec2-bec251299069 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:58.313895171 +0000 UTC m=+18.067519279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs") pod "network-metrics-daemon-tbgzs" (UID: "7b02d944-b4ba-440f-8ec2-bec251299069") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:50.417683 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:50.415406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:50.417683 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.415675    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:50.417683 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.415753    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret podName:0b32c671-b247-4256-878e-a286c276b9ca nodeName:}" failed. No retries permitted until 2026-09-15 22:27:58.415732281 +0000 UTC m=+18.169356410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret") pod "global-pull-secret-syncer-dssx6" (UID: "0b32c671-b247-4256-878e-a286c276b9ca") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:50.757850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:50.757807    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:50.758352 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.757912    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:27:50.758352 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:50.758315    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:50.758465 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:50.758435    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:27:51.756882 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:51.756830    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:51.757095 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:51.757070    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:27:52.756693 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:52.756652    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:52.757222 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:52.756805    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:27:52.757222 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:52.756660    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:52.757222 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:52.756939    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:27:53.757136 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:53.757084    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:53.757677 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:53.757256    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:27:54.756958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:54.756925    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:54.756958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:54.756942    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:54.757225 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:54.757051    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:27:54.757225 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:54.757187    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:27:55.757459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:55.757425    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:55.757880 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:55.757531    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:27:56.757540 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:56.757498    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:56.757999 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:56.757637    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:27:56.757999 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:56.757688    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:56.757999 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:56.757785    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:27:57.756681 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:57.756643    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:57.756878 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:57.756780    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:27:58.380651 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:58.380613    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:58.380651 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:58.380659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:58.381175 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.380758    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:58.381175 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.380778    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:58.381175 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.380795    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:58.381175 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.380805    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8z2t8 for pod openshift-network-diagnostics/network-check-target-rc44q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:58.381175 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.380839    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs podName:7b02d944-b4ba-440f-8ec2-bec251299069 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:14.380820106 +0000 UTC m=+34.134444214 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs") pod "network-metrics-daemon-tbgzs" (UID: "7b02d944-b4ba-440f-8ec2-bec251299069") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:58.381175 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.380861    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8 podName:bae50b7e-fb4b-4da6-b71e-c824b2b6d295 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:14.380850213 +0000 UTC m=+34.134474319 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8z2t8" (UniqueName: "kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8") pod "network-check-target-rc44q" (UID: "bae50b7e-fb4b-4da6-b71e-c824b2b6d295") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:58.482082 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:58.482035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:58.482268 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.482210    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:58.482325 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.482289    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret podName:0b32c671-b247-4256-878e-a286c276b9ca nodeName:}" failed. No retries permitted until 2026-09-15 22:28:14.482269162 +0000 UTC m=+34.235893281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret") pod "global-pull-secret-syncer-dssx6" (UID: "0b32c671-b247-4256-878e-a286c276b9ca") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:58.757554 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:58.757517    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:27:58.757724 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.757663    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:27:58.757873 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:58.757517    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:27:58.758003 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:58.757970    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:27:59.756785 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:27:59.756744    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:27:59.757268 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:27:59.756881    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:00.757741 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:00.757707    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:00.758187 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:00.757885    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:00.758187 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:00.757934    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:00.758187 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:00.758046    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:01.756943 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:01.756872    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:01.757242 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:01.757176    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:02.758469 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.757422    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:02.758469 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:02.757587    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:02.758469 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.757671    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:02.758469 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:02.757764    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:02.862665 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.862452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-77pzk" event={"ID":"d191e41f-4cad-467f-9bdc-86a9b2d61d19","Type":"ContainerStarted","Data":"08af6978c3578ff8fa646711d46864dfb8cc7d433ae85423e9ee7a3699f82c53"}
Sep 15 22:28:02.864971 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.864939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" event={"ID":"59d2402a-a6ba-4c47-bbbd-7d70390292a9","Type":"ContainerStarted","Data":"533c736ac8fdcb26d57df0d5c8be7d276402f10154a0c8f9976689b2af182ba0"}
Sep 15 22:28:02.866666 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.866635    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m4l8b" event={"ID":"0d8ec9c1-59c0-459e-b19b-b86f00b0b55f","Type":"ContainerStarted","Data":"157f35d22212ef4b1e06e5d93044972db101a3247870d3d164087e57ed074bd2"}
Sep 15 22:28:02.868311 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.868283    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xpwtg" event={"ID":"11f3daa5-e359-45dd-8ed1-38ec9eb21ec8","Type":"ContainerStarted","Data":"10f9bf660ee0d1d6cb06d7d4fddfe9daed03646c33d8f01f7c4a33f1a99ae30e"}
Sep 15 22:28:02.870172 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.870148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx" event={"ID":"c592487e-05b6-445b-ba50-08f1899afeaa","Type":"ContainerStarted","Data":"6ba736305df5cfb2fa0e42632cc00bc6f2ef3c184f647fb98e1071c6947effc3"}
Sep 15 22:28:02.872017 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.871798    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gd8k6" event={"ID":"dca0482c-e29e-4e5b-b0e5-0e069d81fdba","Type":"ContainerStarted","Data":"1de087f7e771dc528cd87de5a5fe3dab8372b4e79215527028f162632eaaa2dc"}
Sep 15 22:28:02.873644 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.873618    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c1381d9-2208-4034-89c3-458f4f74eb07" containerID="5b488e3af41f23954caf5db0c364d78dd74b9979d0e23c3de3dd168adb8ac8be" exitCode=0
Sep 15 22:28:02.873753 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.873717    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slcfb" event={"ID":"7c1381d9-2208-4034-89c3-458f4f74eb07","Type":"ContainerDied","Data":"5b488e3af41f23954caf5db0c364d78dd74b9979d0e23c3de3dd168adb8ac8be"}
Sep 15 22:28:02.877499 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.877474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zmr42" event={"ID":"002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc","Type":"ContainerStarted","Data":"6b887aedc12e26656a3004765c332f477d55dbf827db33a552e93bdbae17b733"}
Sep 15 22:28:02.881921 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.881872    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-77pzk" podStartSLOduration=3.064606566 podStartE2EDuration="21.881855396s" totalImagesPullingTime="18.81724883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:02.881802619 +0000 UTC m=+22.635426747" watchObservedRunningTime="2026-09-15 22:28:02.881855396 +0000 UTC m=+22.635479523"
Sep 15 22:28:02.903327 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.903281    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m4l8b" podStartSLOduration=6.427800608 podStartE2EDuration="21.903264262s" totalImagesPullingTime="15.475463654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:02.902833619 +0000 UTC m=+22.656457746" watchObservedRunningTime="2026-09-15 22:28:02.903264262 +0000 UTC m=+22.656888435"
Sep 15 22:28:02.954165 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:02.954088    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zmr42" podStartSLOduration=3.0684814 podStartE2EDuration="21.954072463s" totalImagesPullingTime="18.885591063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:02.953457065 +0000 UTC m=+22.707081192" watchObservedRunningTime="2026-09-15 22:28:02.954072463 +0000 UTC m=+22.707696589"
Sep 15 22:28:03.003965 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.003901    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gd8k6" podStartSLOduration=3.179596696 podStartE2EDuration="22.003882799s" totalImagesPullingTime="18.824286103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:02.976401486 +0000 UTC m=+22.730025613" watchObservedRunningTime="2026-09-15 22:28:03.003882799 +0000 UTC m=+22.757506925"
Sep 15 22:28:03.034250 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.034223    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:28:03.035080 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.035061    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:28:03.059688 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.059637    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xpwtg" podStartSLOduration=3.217218195 podStartE2EDuration="22.059619237s" totalImagesPullingTime="18.842401042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:03.004582578 +0000 UTC m=+22.758206704" watchObservedRunningTime="2026-09-15 22:28:03.059619237 +0000 UTC m=+22.813243364"
Sep 15 22:28:03.756852 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.756661    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:03.757218 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:03.756927    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:03.883906 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.883855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" event={"ID":"59d2402a-a6ba-4c47-bbbd-7d70390292a9","Type":"ContainerStarted","Data":"476af56d43fad29e54172fc28614335b3aec0ca18595be61cf42866ba6233160"}
Sep 15 22:28:03.884808 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.883972    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" event={"ID":"59d2402a-a6ba-4c47-bbbd-7d70390292a9","Type":"ContainerStarted","Data":"fad629073db739c1e67bb0bb037789702f586ba27f17a44ecd08fe210164e38b"}
Sep 15 22:28:03.884808 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.884025    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" event={"ID":"59d2402a-a6ba-4c47-bbbd-7d70390292a9","Type":"ContainerStarted","Data":"0e7114be979399a84d82e2d12e04c319060678d72e8ac7b0b2b7d90560618b64"}
Sep 15 22:28:03.884808 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.884066    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" event={"ID":"59d2402a-a6ba-4c47-bbbd-7d70390292a9","Type":"ContainerStarted","Data":"bfae25b2923dbabcd0d12759cd5b63bc6c2e327cbcc315b5d3863959cda942c4"}
Sep 15 22:28:03.884808 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.884161    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:28:03.884808 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.884201    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" event={"ID":"59d2402a-a6ba-4c47-bbbd-7d70390292a9","Type":"ContainerStarted","Data":"a0c0b6dbc42da9f4a30af7d10c93afd1e78cefc6757f4250a3155678980fc039"}
Sep 15 22:28:03.884808 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:03.884662    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m4l8b"
Sep 15 22:28:04.071050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:04.071022    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 22:28:04.700152 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:04.700027    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T22:28:04.071039194Z","UUID":"18dacd46-61a9-45ce-baa4-65e79c6a9500","Handler":null,"Name":"","Endpoint":""}
Sep 15 22:28:04.702005 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:04.701975    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 22:28:04.702175 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:04.702016    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 22:28:04.757412 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:04.757378    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:04.757575 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:04.757378    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:04.757575 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:04.757517    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:04.757674 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:04.757577    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:04.889036 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:04.888778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vz4vb" event={"ID":"9712cd0c-ad1a-4b5a-aef5-693c83bd051d","Type":"ContainerStarted","Data":"c15add220d077caa74e1af6dc52320058114b59c9dd13b67db7a0c54b687484f"}
Sep 15 22:28:04.891216 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:04.891185    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx" event={"ID":"c592487e-05b6-445b-ba50-08f1899afeaa","Type":"ContainerStarted","Data":"60324c6ce49c6dace431fa6bc5941023930ae0c6014005ce896110288571da38"}
Sep 15 22:28:04.907502 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:04.907447    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vz4vb" podStartSLOduration=8.424868979 podStartE2EDuration="23.9074249s" totalImagesPullingTime="15.482555921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:04.906942441 +0000 UTC m=+24.660566569" watchObservedRunningTime="2026-09-15 22:28:04.9074249 +0000 UTC m=+24.661049028"
Sep 15 22:28:05.756790 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:05.756753    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:05.757006 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:05.756885    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:05.896494 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:05.896454    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx" event={"ID":"c592487e-05b6-445b-ba50-08f1899afeaa","Type":"ContainerStarted","Data":"705bc031671f2e2f8760c4dfafae91a8683ae3b4039b46baee8efb2e3fb257ce"}
Sep 15 22:28:05.900673 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:05.900635    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" event={"ID":"59d2402a-a6ba-4c47-bbbd-7d70390292a9","Type":"ContainerStarted","Data":"b6c3037b889156c140a0898857e587459af85a47fc8a34fc3fdb7be406a3b7e6"}
Sep 15 22:28:05.918353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:05.918296    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bglx" podStartSLOduration=2.969395097 podStartE2EDuration="23.918278257s" totalImagesPullingTime="20.94888316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:05.91664204 +0000 UTC m=+25.670266171" watchObservedRunningTime="2026-09-15 22:28:05.918278257 +0000 UTC m=+25.671902384"
Sep 15 22:28:06.757338 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:06.757303    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:06.757526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:06.757303    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:06.757526 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:06.757425    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:06.757635 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:06.757519    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:07.756997 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:07.756963    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:07.757565 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:07.757069    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:07.907717 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:07.907526    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slcfb" event={"ID":"7c1381d9-2208-4034-89c3-458f4f74eb07","Type":"ContainerStarted","Data":"a5cc5eb524073a7ae3cc603cc3ac09731d5c4ddb66e1040c383cc326afe742ad"}
Sep 15 22:28:07.911329 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:07.911240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" event={"ID":"59d2402a-a6ba-4c47-bbbd-7d70390292a9","Type":"ContainerStarted","Data":"f665b0e23808527bdae6bd3f48459bc1b93abcd4b7cb51e2f909f885242635f7"}
Sep 15 22:28:07.911474 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:07.911426    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:28:07.911549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:07.911529    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:28:07.911600 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:07.911584    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:28:07.927272 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:07.927247    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:28:07.927393 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:07.927351    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:28:07.963692 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:07.963636    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7" podStartSLOduration=8.068639069 podStartE2EDuration="26.963619286s" totalImagesPullingTime="18.894980217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:07.963408192 +0000 UTC m=+27.717032324" watchObservedRunningTime="2026-09-15 22:28:07.963619286 +0000 UTC m=+27.717243412"
Sep 15 22:28:08.756745 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:08.756706    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:08.756983 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:08.756825    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:08.756983 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:08.756858    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:08.756983 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:08.756907    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:08.915772 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:08.915737    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c1381d9-2208-4034-89c3-458f4f74eb07" containerID="a5cc5eb524073a7ae3cc603cc3ac09731d5c4ddb66e1040c383cc326afe742ad" exitCode=0
Sep 15 22:28:08.915959 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:08.915830    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slcfb" event={"ID":"7c1381d9-2208-4034-89c3-458f4f74eb07","Type":"ContainerDied","Data":"a5cc5eb524073a7ae3cc603cc3ac09731d5c4ddb66e1040c383cc326afe742ad"}
Sep 15 22:28:09.757640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:09.757455    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:09.758035 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:09.757715    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:10.757936 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:10.757899    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:10.758376 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:10.757978    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:10.758376 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:10.757997    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:10.758376 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:10.758106    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:10.924434 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:10.924400    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c1381d9-2208-4034-89c3-458f4f74eb07" containerID="c383e489b43ca9e00466495c7b9f35a57f77df561057be71a8dfda706da92880" exitCode=0
Sep 15 22:28:10.924604 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:10.924458    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slcfb" event={"ID":"7c1381d9-2208-4034-89c3-458f4f74eb07","Type":"ContainerDied","Data":"c383e489b43ca9e00466495c7b9f35a57f77df561057be71a8dfda706da92880"}
Sep 15 22:28:11.756753 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:11.756711    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:11.756949 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:11.756836    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:12.756710 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:12.756679    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:12.757147 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:12.756727    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:12.757147 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:12.756819    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:12.757147 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:12.756926    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:12.931856 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:12.931820    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c1381d9-2208-4034-89c3-458f4f74eb07" containerID="40eef328a61686c15794570f029c4a9187071ac8f011c755383cca4bab1da41d" exitCode=0
Sep 15 22:28:12.932007 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:12.931869    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slcfb" event={"ID":"7c1381d9-2208-4034-89c3-458f4f74eb07","Type":"ContainerDied","Data":"40eef328a61686c15794570f029c4a9187071ac8f011c755383cca4bab1da41d"}
Sep 15 22:28:13.756836 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:13.756789    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:13.757361 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:13.756924    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:14.399205 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:14.399158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:14.399391 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:14.399228    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:14.399391 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.399341    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:28:14.399391 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.399362    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:28:14.399391 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.399378    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:28:14.399391 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.399388    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8z2t8 for pod openshift-network-diagnostics/network-check-target-rc44q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:28:14.399609 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.399420    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs podName:7b02d944-b4ba-440f-8ec2-bec251299069 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:46.399403695 +0000 UTC m=+66.153027801 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs") pod "network-metrics-daemon-tbgzs" (UID: "7b02d944-b4ba-440f-8ec2-bec251299069") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:28:14.399609 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.399441    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8 podName:bae50b7e-fb4b-4da6-b71e-c824b2b6d295 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:46.399430958 +0000 UTC m=+66.153055064 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8z2t8" (UniqueName: "kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8") pod "network-check-target-rc44q" (UID: "bae50b7e-fb4b-4da6-b71e-c824b2b6d295") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:28:14.499590 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:14.499557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:14.499769 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.499675    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:28:14.499769 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.499729    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret podName:0b32c671-b247-4256-878e-a286c276b9ca nodeName:}" failed. No retries permitted until 2026-09-15 22:28:46.499715315 +0000 UTC m=+66.253339434 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret") pod "global-pull-secret-syncer-dssx6" (UID: "0b32c671-b247-4256-878e-a286c276b9ca") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:28:14.756880 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:14.756843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:14.757339 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:14.756851    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:14.757339 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.757064    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:14.757339 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:14.757067    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:15.757544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:15.757506    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:15.758026 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:15.757625    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:16.756691 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:16.756655    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:16.756877 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:16.756786    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:16.756877 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:16.756846    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:16.757005 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:16.756940    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:17.756932 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:17.756901    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:17.757331 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:17.757034    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:17.946052 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:17.946022    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c1381d9-2208-4034-89c3-458f4f74eb07" containerID="321d2c5d0cdf1c2ef0a3a647088d3b65eb4589c88e03bf469a71ab2ec53f475c" exitCode=0
Sep 15 22:28:17.946237 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:17.946081    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slcfb" event={"ID":"7c1381d9-2208-4034-89c3-458f4f74eb07","Type":"ContainerDied","Data":"321d2c5d0cdf1c2ef0a3a647088d3b65eb4589c88e03bf469a71ab2ec53f475c"}
Sep 15 22:28:18.756648 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:18.756615    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:18.756814 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:18.756758    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:18.756814 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:18.756806    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:18.756899 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:18.756888    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:18.951763 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:18.951727    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c1381d9-2208-4034-89c3-458f4f74eb07" containerID="e7fe202df81bf1caa5fd0aa9fb45608ab633dce0f644a32ba568a929fc446542" exitCode=0
Sep 15 22:28:18.952205 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:18.951784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slcfb" event={"ID":"7c1381d9-2208-4034-89c3-458f4f74eb07","Type":"ContainerDied","Data":"e7fe202df81bf1caa5fd0aa9fb45608ab633dce0f644a32ba568a929fc446542"}
Sep 15 22:28:19.757543 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:19.757502    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:19.757747 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:19.757606    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:19.957649 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:19.957614    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-slcfb" event={"ID":"7c1381d9-2208-4034-89c3-458f4f74eb07","Type":"ContainerStarted","Data":"9faa321ccb950d11bdcf69bbe0f2a0647abbe0fa859a8fadc41edc63f92aa9eb"}
Sep 15 22:28:20.758327 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:20.758284    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:20.758504 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:20.758383    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:20.758504 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:20.758472    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:20.758611 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:20.758590    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:21.756815 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:21.756772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:21.757249 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:21.756879    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:22.757630 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:22.757579    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:22.758246 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:22.757585    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:22.758246 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:22.757777    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:22.758246 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:22.757873    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:23.756708 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:23.756531    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:23.756934 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:23.756822    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:23.940371 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:23.940246    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-slcfb" podStartSLOduration=12.880965761 podStartE2EDuration="42.94022237s" totalImagesPullingTime="30.059256609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:19.988676765 +0000 UTC m=+39.742300896" watchObservedRunningTime="2026-09-15 22:28:23.94022237 +0000 UTC m=+43.693846497"
Sep 15 22:28:23.941496 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:23.941467    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dssx6"]
Sep 15 22:28:23.941698 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:23.941584    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:23.941766 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:23.941701    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:23.943573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:23.943542    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rc44q"]
Sep 15 22:28:23.949356 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:23.949329    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tbgzs"]
Sep 15 22:28:23.949495 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:23.949452    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:23.949574 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:23.949555    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:23.967235 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:23.967207    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:23.967392 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:23.967317    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:25.756911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:25.756863    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:25.757419 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:25.756915    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:25.757419 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:25.756872    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:25.757419 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:25.757004    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:25.757419 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:25.757071    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:25.757419 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:25.757169    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:27.757235 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:27.757192    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:27.757729 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:27.757192    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:27.757729 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:27.757336    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dssx6" podUID="0b32c671-b247-4256-878e-a286c276b9ca"
Sep 15 22:28:27.757729 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:27.757192    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:27.757729 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:27.757367    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc44q" podUID="bae50b7e-fb4b-4da6-b71e-c824b2b6d295"
Sep 15 22:28:27.757729 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:27.757462    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tbgzs" podUID="7b02d944-b4ba-440f-8ec2-bec251299069"
Sep 15 22:28:28.507193 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.507165    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-247.ec2.internal" event="NodeReady"
Sep 15 22:28:28.507376 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.507292    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 22:28:28.559673 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.559637    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6qwth"]
Sep 15 22:28:28.577044 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.577005    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pmjjm"]
Sep 15 22:28:28.577232 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.577139    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6qwth"
Sep 15 22:28:28.580232 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.580207    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vrlmr\""
Sep 15 22:28:28.580374 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.580349    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 22:28:28.580685 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.580664    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 22:28:28.580769 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.580665    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 22:28:28.599202 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.599168    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6qwth"]
Sep 15 22:28:28.599202 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.599211    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-crlld"]
Sep 15 22:28:28.599424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.599353    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.607154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.603180    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 22:28:28.607154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.603498    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 22:28:28.607154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.603700    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 22:28:28.607154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.604471    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-44p4v\""
Sep 15 22:28:28.607154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.605357    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 22:28:28.627345 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.627300    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pmjjm"]
Sep 15 22:28:28.627518 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.627352    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.627518 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.627403    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-crlld"]
Sep 15 22:28:28.630293 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.630269    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8pmw9\""
Sep 15 22:28:28.630426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.630338    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 22:28:28.630905 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.630891    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 22:28:28.695459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.695418    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fc4065e-13df-4e73-906a-55b17f8ab389-data-volume\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.695459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.695459    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6fc4065e-13df-4e73-906a-55b17f8ab389-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.695676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.695506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6fc4065e-13df-4e73-906a-55b17f8ab389-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.695676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.695553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6fc4065e-13df-4e73-906a-55b17f8ab389-crio-socket\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.695676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.695575    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f46c2b32-1ee3-4064-9065-911fdae76f42-cert\") pod \"ingress-canary-6qwth\" (UID: \"f46c2b32-1ee3-4064-9065-911fdae76f42\") " pod="openshift-ingress-canary/ingress-canary-6qwth"
Sep 15 22:28:28.695676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.695608    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4nl6\" (UniqueName: \"kubernetes.io/projected/f46c2b32-1ee3-4064-9065-911fdae76f42-kube-api-access-t4nl6\") pod \"ingress-canary-6qwth\" (UID: \"f46c2b32-1ee3-4064-9065-911fdae76f42\") " pod="openshift-ingress-canary/ingress-canary-6qwth"
Sep 15 22:28:28.695794 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.695688    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8g7f\" (UniqueName: \"kubernetes.io/projected/6fc4065e-13df-4e73-906a-55b17f8ab389-kube-api-access-f8g7f\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.797083 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.796994    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6fc4065e-13df-4e73-906a-55b17f8ab389-crio-socket\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.797083 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f46c2b32-1ee3-4064-9065-911fdae76f42-cert\") pod \"ingress-canary-6qwth\" (UID: \"f46c2b32-1ee3-4064-9065-911fdae76f42\") " pod="openshift-ingress-canary/ingress-canary-6qwth"
Sep 15 22:28:28.797083 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-config-volume\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.797689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797144    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66xlf\" (UniqueName: \"kubernetes.io/projected/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-kube-api-access-66xlf\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.797689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4nl6\" (UniqueName: \"kubernetes.io/projected/f46c2b32-1ee3-4064-9065-911fdae76f42-kube-api-access-t4nl6\") pod \"ingress-canary-6qwth\" (UID: \"f46c2b32-1ee3-4064-9065-911fdae76f42\") " pod="openshift-ingress-canary/ingress-canary-6qwth"
Sep 15 22:28:28.797689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8g7f\" (UniqueName: \"kubernetes.io/projected/6fc4065e-13df-4e73-906a-55b17f8ab389-kube-api-access-f8g7f\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.797689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6fc4065e-13df-4e73-906a-55b17f8ab389-crio-socket\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.797689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-tmp-dir\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.797689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fc4065e-13df-4e73-906a-55b17f8ab389-data-volume\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.797689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797367    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6fc4065e-13df-4e73-906a-55b17f8ab389-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.797689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797383    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-metrics-tls\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.797689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6fc4065e-13df-4e73-906a-55b17f8ab389-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.798028 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797729    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fc4065e-13df-4e73-906a-55b17f8ab389-data-volume\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.798028 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.797972    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6fc4065e-13df-4e73-906a-55b17f8ab389-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.801126 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.801083    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6fc4065e-13df-4e73-906a-55b17f8ab389-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.801226 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.801138    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f46c2b32-1ee3-4064-9065-911fdae76f42-cert\") pod \"ingress-canary-6qwth\" (UID: \"f46c2b32-1ee3-4064-9065-911fdae76f42\") " pod="openshift-ingress-canary/ingress-canary-6qwth"
Sep 15 22:28:28.811273 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.811247    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8g7f\" (UniqueName: \"kubernetes.io/projected/6fc4065e-13df-4e73-906a-55b17f8ab389-kube-api-access-f8g7f\") pod \"insights-runtime-extractor-pmjjm\" (UID: \"6fc4065e-13df-4e73-906a-55b17f8ab389\") " pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.811475 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.811453    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4nl6\" (UniqueName: \"kubernetes.io/projected/f46c2b32-1ee3-4064-9065-911fdae76f42-kube-api-access-t4nl6\") pod \"ingress-canary-6qwth\" (UID: \"f46c2b32-1ee3-4064-9065-911fdae76f42\") " pod="openshift-ingress-canary/ingress-canary-6qwth"
Sep 15 22:28:28.831399 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.831368    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7767f5798b-8nqkg"]
Sep 15 22:28:28.863991 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.863957    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7767f5798b-8nqkg"]
Sep 15 22:28:28.864173 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.864063    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:28.867108 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.867072    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 22:28:28.867238 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.867181    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4z6d6\""
Sep 15 22:28:28.867238 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.867188    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 22:28:28.867351 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.867333    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 22:28:28.867472 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.867459    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 22:28:28.867664 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.867647    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 22:28:28.867713 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.867673    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 22:28:28.867713 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.867691    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 22:28:28.875396 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.875375    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 15 22:28:28.888114 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.888082    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6qwth"
Sep 15 22:28:28.898424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.898397    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-tmp-dir\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.898536 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.898431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-metrics-tls\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.898536 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.898466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-config-volume\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.898536 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.898492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66xlf\" (UniqueName: \"kubernetes.io/projected/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-kube-api-access-66xlf\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.898730 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.898710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-tmp-dir\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.899050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.899031    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-config-volume\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.900752 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.900733    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-metrics-tls\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.907214 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.907191    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66xlf\" (UniqueName: \"kubernetes.io/projected/1604b4fa-cf86-41f4-9a0d-0db55bad3f65-kube-api-access-66xlf\") pod \"dns-default-crlld\" (UID: \"1604b4fa-cf86-41f4-9a0d-0db55bad3f65\") " pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:28.912121 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.912082    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pmjjm"
Sep 15 22:28:28.936984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.936951    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:29.003247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:28.999976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjt4m\" (UniqueName: \"kubernetes.io/projected/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-kube-api-access-hjt4m\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.003247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.000300    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-serving-cert\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.003247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.000332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-trusted-ca-bundle\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.003247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.000357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-oauth-serving-cert\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.003247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.000397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-oauth-config\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.003247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.000422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-service-ca\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.003247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.000534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-config\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.101864 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.101726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjt4m\" (UniqueName: \"kubernetes.io/projected/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-kube-api-access-hjt4m\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.102015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.101877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-serving-cert\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.102015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.101918    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-trusted-ca-bundle\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.102015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.101946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-oauth-serving-cert\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.102015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.101986    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-oauth-config\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.102015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.102009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-service-ca\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.102313 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.102217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-config\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.102855 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.102826    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-service-ca\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.102966 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.102934    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-oauth-serving-cert\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.102966 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.102943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-config\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.103448 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.103427    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-trusted-ca-bundle\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.104818 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.104801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-serving-cert\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.104901 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.104830    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-oauth-config\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.112002 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.111977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjt4m\" (UniqueName: \"kubernetes.io/projected/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-kube-api-access-hjt4m\") pod \"console-7767f5798b-8nqkg\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") " pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.175053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.175020    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:29.209030 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.208998    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6qwth"]
Sep 15 22:28:29.228435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.228398    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pmjjm"]
Sep 15 22:28:29.232325 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:29.232296    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc4065e_13df_4e73_906a_55b17f8ab389.slice/crio-e78eced48e3c322733fe7e501049753d025ee560fdd604b675c13a5a311707fb WatchSource:0}: Error finding container e78eced48e3c322733fe7e501049753d025ee560fdd604b675c13a5a311707fb: Status 404 returned error can't find the container with id e78eced48e3c322733fe7e501049753d025ee560fdd604b675c13a5a311707fb
Sep 15 22:28:29.248840 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.248754    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-crlld"]
Sep 15 22:28:29.252648 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:29.252613    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1604b4fa_cf86_41f4_9a0d_0db55bad3f65.slice/crio-817cc3e7535794a7e6eb8664715ddb86bcf14ad2c8f08c93ed7513ca7e7b619b WatchSource:0}: Error finding container 817cc3e7535794a7e6eb8664715ddb86bcf14ad2c8f08c93ed7513ca7e7b619b: Status 404 returned error can't find the container with id 817cc3e7535794a7e6eb8664715ddb86bcf14ad2c8f08c93ed7513ca7e7b619b
Sep 15 22:28:29.470450 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.470417    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7767f5798b-8nqkg"]
Sep 15 22:28:29.474313 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:29.474281    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f82e78_2ca9_47d3_984d_e4f31ab9cf09.slice/crio-31834d985df2f8181591461759d1ceee47b3beb626ae203b6143bca7aded4b50 WatchSource:0}: Error finding container 31834d985df2f8181591461759d1ceee47b3beb626ae203b6143bca7aded4b50: Status 404 returned error can't find the container with id 31834d985df2f8181591461759d1ceee47b3beb626ae203b6143bca7aded4b50
Sep 15 22:28:29.757264 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.757230    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:29.757473 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.757231    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:29.757473 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.757241    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:29.761856 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.761829    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-29265\""
Sep 15 22:28:29.762011 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.761829    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 22:28:29.762011 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.761874    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 22:28:29.762011 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.761914    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 22:28:29.762011 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.761992    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 22:28:29.762295 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.762271    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fznt9\""
Sep 15 22:28:29.984314 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.984273    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7767f5798b-8nqkg" event={"ID":"50f82e78-2ca9-47d3-984d-e4f31ab9cf09","Type":"ContainerStarted","Data":"31834d985df2f8181591461759d1ceee47b3beb626ae203b6143bca7aded4b50"}
Sep 15 22:28:29.985543 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.985514    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-crlld" event={"ID":"1604b4fa-cf86-41f4-9a0d-0db55bad3f65","Type":"ContainerStarted","Data":"817cc3e7535794a7e6eb8664715ddb86bcf14ad2c8f08c93ed7513ca7e7b619b"}
Sep 15 22:28:29.987839 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.987806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmjjm" event={"ID":"6fc4065e-13df-4e73-906a-55b17f8ab389","Type":"ContainerStarted","Data":"64a2d2f138136a94a57b93a15ee977035030d780e57771e43ebbccfa1fc47fb3"}
Sep 15 22:28:29.987983 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.987867    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmjjm" event={"ID":"6fc4065e-13df-4e73-906a-55b17f8ab389","Type":"ContainerStarted","Data":"e78eced48e3c322733fe7e501049753d025ee560fdd604b675c13a5a311707fb"}
Sep 15 22:28:29.989237 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:29.989205    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6qwth" event={"ID":"f46c2b32-1ee3-4064-9065-911fdae76f42","Type":"ContainerStarted","Data":"6c66af6c8c93370b9bc81308de916c42a1d5fb42d2ebd16f5feb76c6f40d9693"}
Sep 15 22:28:30.996709 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:30.996661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmjjm" event={"ID":"6fc4065e-13df-4e73-906a-55b17f8ab389","Type":"ContainerStarted","Data":"5a39f5d154535f02a6249e9fca3f2544174de920360656c0814101198ebaf4de"}
Sep 15 22:28:33.522387 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.522135    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"]
Sep 15 22:28:33.524740 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.524700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.529385 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.529357    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 22:28:33.529533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.529355    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 15 22:28:33.529738 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.529703    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 22:28:33.529738 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.529728    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 22:28:33.529884 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.529804    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 15 22:28:33.530046 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.530031    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-g55nt\""
Sep 15 22:28:33.536045 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.536017    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"]
Sep 15 22:28:33.642761 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.642672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkznx\" (UniqueName: \"kubernetes.io/projected/99cff14d-1671-4944-aa51-51baab7a2d81-kube-api-access-nkznx\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.642923 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.642765    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99cff14d-1671-4944-aa51-51baab7a2d81-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.642923 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.642834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/99cff14d-1671-4944-aa51-51baab7a2d81-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.643014 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.642912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99cff14d-1671-4944-aa51-51baab7a2d81-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.744153 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.744089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99cff14d-1671-4944-aa51-51baab7a2d81-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.744336 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.744185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/99cff14d-1671-4944-aa51-51baab7a2d81-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.744336 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.744217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99cff14d-1671-4944-aa51-51baab7a2d81-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.744336 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.744249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkznx\" (UniqueName: \"kubernetes.io/projected/99cff14d-1671-4944-aa51-51baab7a2d81-kube-api-access-nkznx\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.745160 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.745136    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99cff14d-1671-4944-aa51-51baab7a2d81-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.748503 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.748465    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99cff14d-1671-4944-aa51-51baab7a2d81-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.748503 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.748499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/99cff14d-1671-4944-aa51-51baab7a2d81-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.753609 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.753574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkznx\" (UniqueName: \"kubernetes.io/projected/99cff14d-1671-4944-aa51-51baab7a2d81-kube-api-access-nkznx\") pod \"prometheus-operator-68bcd7f987-hn2mz\" (UID: \"99cff14d-1671-4944-aa51-51baab7a2d81\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:33.836500 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:33.836456    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"
Sep 15 22:28:34.351339 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:34.351105    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz"]
Sep 15 22:28:34.355367 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:34.355343    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cff14d_1671_4944_aa51_51baab7a2d81.slice/crio-22f18c4082fe24607b363ac1865d57e1ca657b1ab5ca6bdd8d75d9cc5b9b1c16 WatchSource:0}: Error finding container 22f18c4082fe24607b363ac1865d57e1ca657b1ab5ca6bdd8d75d9cc5b9b1c16: Status 404 returned error can't find the container with id 22f18c4082fe24607b363ac1865d57e1ca657b1ab5ca6bdd8d75d9cc5b9b1c16
Sep 15 22:28:35.012031 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.011986    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7767f5798b-8nqkg" event={"ID":"50f82e78-2ca9-47d3-984d-e4f31ab9cf09","Type":"ContainerStarted","Data":"28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070"}
Sep 15 22:28:35.017106 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.014197    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-crlld" event={"ID":"1604b4fa-cf86-41f4-9a0d-0db55bad3f65","Type":"ContainerStarted","Data":"e4176d789771658448f800a5348c947adf3db21715e06705ccb107b130d98269"}
Sep 15 22:28:35.017106 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.014327    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-crlld" event={"ID":"1604b4fa-cf86-41f4-9a0d-0db55bad3f65","Type":"ContainerStarted","Data":"ed55b54b54b70c35985fa9899f13b3b4d7f9d3577b91d8807655f374660e8def"}
Sep 15 22:28:35.017106 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.014427    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:35.019436 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.019405    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmjjm" event={"ID":"6fc4065e-13df-4e73-906a-55b17f8ab389","Type":"ContainerStarted","Data":"288f82e533538429ffbe96e4c0d045f5646b2d8e541888a56bde50b7c4e56eb8"}
Sep 15 22:28:35.021044 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.021018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6qwth" event={"ID":"f46c2b32-1ee3-4064-9065-911fdae76f42","Type":"ContainerStarted","Data":"50475a32263897ae34dab3c094b8581901549ad8809aeb8564572efea3bba66f"}
Sep 15 22:28:35.022609 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.022584    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz" event={"ID":"99cff14d-1671-4944-aa51-51baab7a2d81","Type":"ContainerStarted","Data":"22f18c4082fe24607b363ac1865d57e1ca657b1ab5ca6bdd8d75d9cc5b9b1c16"}
Sep 15 22:28:35.040229 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.040157    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7767f5798b-8nqkg" podStartSLOduration=2.451443743 podStartE2EDuration="7.040141228s" totalImagesPullingTime="4.588697485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:35.038523415 +0000 UTC m=+54.792147541" watchObservedRunningTime="2026-09-15 22:28:35.040141228 +0000 UTC m=+54.793765348"
Sep 15 22:28:35.058118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.057451    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6qwth" podStartSLOduration=2.207753522 podStartE2EDuration="7.057432991s" totalImagesPullingTime="4.849679469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:35.056812634 +0000 UTC m=+54.810436762" watchObservedRunningTime="2026-09-15 22:28:35.057432991 +0000 UTC m=+54.811057117"
Sep 15 22:28:35.076922 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.076867    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pmjjm" podStartSLOduration=2.536927671 podStartE2EDuration="7.076845842s" totalImagesPullingTime="4.539918171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:28:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:35.075818725 +0000 UTC m=+54.829442895" watchObservedRunningTime="2026-09-15 22:28:35.076845842 +0000 UTC m=+54.830469968"
Sep 15 22:28:35.097721 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:35.097663    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-crlld" podStartSLOduration=2.294909422 podStartE2EDuration="7.097641212s" totalImagesPullingTime="4.80273179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:35.096992484 +0000 UTC m=+54.850616611" watchObservedRunningTime="2026-09-15 22:28:35.097641212 +0000 UTC m=+54.851265328"
Sep 15 22:28:36.028427 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:36.028319    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz" event={"ID":"99cff14d-1671-4944-aa51-51baab7a2d81","Type":"ContainerStarted","Data":"1d9bf5e359f4e0e0ba2490a9e7292a25bf5604528202442f91a215dc3a6ae681"}
Sep 15 22:28:36.028427 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:36.028385    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz" event={"ID":"99cff14d-1671-4944-aa51-51baab7a2d81","Type":"ContainerStarted","Data":"09943d248fcc4a1daaf734c8e1b3a4ddb9500d6780134783d7205eae7c6ad140"}
Sep 15 22:28:36.047375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:36.047321    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hn2mz" podStartSLOduration=1.7093468509999998 podStartE2EDuration="3.047306113s" totalImagesPullingTime="1.337959262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:36.045898282 +0000 UTC m=+55.799522409" watchObservedRunningTime="2026-09-15 22:28:36.047306113 +0000 UTC m=+55.800930239"
Sep 15 22:28:37.920531 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.920496    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8vldl"]
Sep 15 22:28:37.922695 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.922676    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:37.927751 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.926514    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 22:28:37.927751 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.926736    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 22:28:37.927751 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.926885    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-99wnf\""
Sep 15 22:28:37.927751 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.927430    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 22:28:37.976335 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.976297    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1686a97a-f7a6-4f90-8afa-082636493d94-root\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:37.976335 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.976343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s2dk\" (UniqueName: \"kubernetes.io/projected/1686a97a-f7a6-4f90-8afa-082636493d94-kube-api-access-5s2dk\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:37.976576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.976363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-wtmp\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:37.976576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.976433    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1686a97a-f7a6-4f90-8afa-082636493d94-sys\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:37.976576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.976464    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-textfile\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:37.976576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.976526    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-accelerators-collector-config\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:37.976576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.976560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1686a97a-f7a6-4f90-8afa-082636493d94-metrics-client-ca\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:37.976790 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.976579    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:37.976790 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:37.976604    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-tls\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.077911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.077850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1686a97a-f7a6-4f90-8afa-082636493d94-sys\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.077911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.077904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-textfile\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078215 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.077977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1686a97a-f7a6-4f90-8afa-082636493d94-sys\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078215 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-accelerators-collector-config\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078215 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078149    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1686a97a-f7a6-4f90-8afa-082636493d94-metrics-client-ca\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078215 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078215 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078204    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-textfile\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078227    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-tls\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1686a97a-f7a6-4f90-8afa-082636493d94-root\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s2dk\" (UniqueName: \"kubernetes.io/projected/1686a97a-f7a6-4f90-8afa-082636493d94-kube-api-access-5s2dk\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-wtmp\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078453 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:38.078339    2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 15 22:28:38.078453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1686a97a-f7a6-4f90-8afa-082636493d94-root\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078453 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:38.078427    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-tls podName:1686a97a-f7a6-4f90-8afa-082636493d94 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:38.578407054 +0000 UTC m=+58.332031173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-tls") pod "node-exporter-8vldl" (UID: "1686a97a-f7a6-4f90-8afa-082636493d94") : secret "node-exporter-tls" not found
Sep 15 22:28:38.078738 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-wtmp\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078738 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1686a97a-f7a6-4f90-8afa-082636493d94-metrics-client-ca\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.078852 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.078813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-accelerators-collector-config\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.081685 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.081661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.090506 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.090481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s2dk\" (UniqueName: \"kubernetes.io/projected/1686a97a-f7a6-4f90-8afa-082636493d94-kube-api-access-5s2dk\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.582640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.582588    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-tls\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.585076 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.585033    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1686a97a-f7a6-4f90-8afa-082636493d94-node-exporter-tls\") pod \"node-exporter-8vldl\" (UID: \"1686a97a-f7a6-4f90-8afa-082636493d94\") " pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.837280 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:38.837177    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8vldl"
Sep 15 22:28:38.884645 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:38.884620    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1686a97a_f7a6_4f90_8afa_082636493d94.slice/crio-6e266423aebd0abc7f90e33a19e4778eada8cd83c5d88ab79a85bdad4896ea09 WatchSource:0}: Error finding container 6e266423aebd0abc7f90e33a19e4778eada8cd83c5d88ab79a85bdad4896ea09: Status 404 returned error can't find the container with id 6e266423aebd0abc7f90e33a19e4778eada8cd83c5d88ab79a85bdad4896ea09
Sep 15 22:28:39.039369 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:39.039330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8vldl" event={"ID":"1686a97a-f7a6-4f90-8afa-082636493d94","Type":"ContainerStarted","Data":"6e266423aebd0abc7f90e33a19e4778eada8cd83c5d88ab79a85bdad4896ea09"}
Sep 15 22:28:39.175291 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:39.175195    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:39.175291 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:39.175267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:39.180289 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:39.180258    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:39.939549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:39.939516    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k6vk7"
Sep 15 22:28:40.048564 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.048471    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:28:40.965044 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.964987    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"]
Sep 15 22:28:40.968005 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.967986    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:40.970947 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.970922    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e4mk5qudmi8qf\""
Sep 15 22:28:40.970947 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.970937    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 15 22:28:40.971379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.971361    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 15 22:28:40.971379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.971369    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 15 22:28:40.971529 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.971486    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 15 22:28:40.971859 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.971843    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 15 22:28:40.971918 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.971880    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lbptk\""
Sep 15 22:28:40.984011 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:40.983980    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"]
Sep 15 22:28:41.002248 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.002213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.002248 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.002249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-grpc-tls\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.002488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.002286    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.002488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.002344    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.002488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.002365    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6rtk\" (UniqueName: \"kubernetes.io/projected/287cae11-8551-4505-aebe-9f1623157a9a-kube-api-access-f6rtk\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.002488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.002380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/287cae11-8551-4505-aebe-9f1623157a9a-metrics-client-ca\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.002488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.002428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-tls\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.002658 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.002493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.051275 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.051242    2578 generic.go:356] "Generic (PLEG): container finished" podID="1686a97a-f7a6-4f90-8afa-082636493d94" containerID="2d33a70ea25a6a8a7ff0fab7edbfa8345d6a5d3bd527156cb73acae84e7a2ae1" exitCode=0
Sep 15 22:28:41.051731 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.051322    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8vldl" event={"ID":"1686a97a-f7a6-4f90-8afa-082636493d94","Type":"ContainerDied","Data":"2d33a70ea25a6a8a7ff0fab7edbfa8345d6a5d3bd527156cb73acae84e7a2ae1"}
Sep 15 22:28:41.102879 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.102850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.103015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.102904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6rtk\" (UniqueName: \"kubernetes.io/projected/287cae11-8551-4505-aebe-9f1623157a9a-kube-api-access-f6rtk\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.103241 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.103212    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/287cae11-8551-4505-aebe-9f1623157a9a-metrics-client-ca\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.103324 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.103294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-tls\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.103388 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.103343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.103486 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.103470    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.103546 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.103501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-grpc-tls\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.103633 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.103603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.103916 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.103890    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/287cae11-8551-4505-aebe-9f1623157a9a-metrics-client-ca\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.106955 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.106920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.107119 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.106958    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-tls\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.107119 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.107053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.107673 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.107647    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.107811 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.107790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.107882 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.107861    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/287cae11-8551-4505-aebe-9f1623157a9a-secret-grpc-tls\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.110921 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.110895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6rtk\" (UniqueName: \"kubernetes.io/projected/287cae11-8551-4505-aebe-9f1623157a9a-kube-api-access-f6rtk\") pod \"thanos-querier-8cd6b5b54-rspv2\" (UID: \"287cae11-8551-4505-aebe-9f1623157a9a\") " pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.277154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.277034    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:41.559469 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:41.559375    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"]
Sep 15 22:28:41.563342 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:41.563317    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod287cae11_8551_4505_aebe_9f1623157a9a.slice/crio-7d7dc22d0a4b33181e6a956ee4f9dbc1b16e816334fd6377f291431a185eb025 WatchSource:0}: Error finding container 7d7dc22d0a4b33181e6a956ee4f9dbc1b16e816334fd6377f291431a185eb025: Status 404 returned error can't find the container with id 7d7dc22d0a4b33181e6a956ee4f9dbc1b16e816334fd6377f291431a185eb025
Sep 15 22:28:42.056752 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.056716    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8vldl" event={"ID":"1686a97a-f7a6-4f90-8afa-082636493d94","Type":"ContainerStarted","Data":"0b7b4cdec6f92ec24b8f29bc188c23198c53c568bb1a5775d7c877ac01e16ffa"}
Sep 15 22:28:42.057246 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.056780    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8vldl" event={"ID":"1686a97a-f7a6-4f90-8afa-082636493d94","Type":"ContainerStarted","Data":"6b6990ae196dc1b2c0161153541bb2705e311d971ce8b4215cc9180a34255801"}
Sep 15 22:28:42.057932 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.057901    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2" event={"ID":"287cae11-8551-4505-aebe-9f1623157a9a","Type":"ContainerStarted","Data":"7d7dc22d0a4b33181e6a956ee4f9dbc1b16e816334fd6377f291431a185eb025"}
Sep 15 22:28:42.077700 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.077649    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8vldl" podStartSLOduration=3.589464208 podStartE2EDuration="5.077630892s" totalImagesPullingTime="1.423988585s" totalInitContainerRuntime="64.178099ms" isStatefulPod=true podCreationTimestamp="2026-09-15 22:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:42.07689266 +0000 UTC m=+61.830516791" watchObservedRunningTime="2026-09-15 22:28:42.077630892 +0000 UTC m=+61.831255019"
Sep 15 22:28:42.372884 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.372794    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7cc447d874-cknv6"]
Sep 15 22:28:42.374989 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.374956    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.378457 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.378139    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 15 22:28:42.378457 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.378197    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-m8mbh\""
Sep 15 22:28:42.378457 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.378215    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 15 22:28:42.379573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.379431    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 15 22:28:42.379573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.379434    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 15 22:28:42.379808 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.379785    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cduevjto93sh7\""
Sep 15 22:28:42.386142 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.386061    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7cc447d874-cknv6"]
Sep 15 22:28:42.415035 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.414998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-audit-log\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.415229 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.415050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-client-ca-bundle\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.415229 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.415151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-secret-metrics-server-client-certs\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.415229 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.415185    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k452j\" (UniqueName: \"kubernetes.io/projected/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-kube-api-access-k452j\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.415355 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.415251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.415355 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.415292    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-secret-metrics-server-tls\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.415355 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.415312    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-metrics-server-audit-profiles\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.516392 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.516355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-audit-log\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.516567 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.516409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-client-ca-bundle\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.516567 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.516528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-secret-metrics-server-client-certs\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.516567 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.516547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k452j\" (UniqueName: \"kubernetes.io/projected/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-kube-api-access-k452j\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.516698 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.516596    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.516698 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.516640    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-secret-metrics-server-tls\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.516698 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.516668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-metrics-server-audit-profiles\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.516976 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.516851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-audit-log\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.517413 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.517373    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.517673 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.517650    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-metrics-server-audit-profiles\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.519661 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.519636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-secret-metrics-server-client-certs\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.519661 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.519650    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-secret-metrics-server-tls\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.519812 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.519716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-client-ca-bundle\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.525796 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.525772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k452j\" (UniqueName: \"kubernetes.io/projected/03fd2ff5-486f-495c-b00a-dc31cc78c6f3-kube-api-access-k452j\") pod \"metrics-server-7cc447d874-cknv6\" (UID: \"03fd2ff5-486f-495c-b00a-dc31cc78c6f3\") " pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.666430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.666342    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"]
Sep 15 22:28:42.669278 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.669245    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:42.672248 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.672222    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 15 22:28:42.672532 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.672511    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jplwk\""
Sep 15 22:28:42.680228 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.680203    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"]
Sep 15 22:28:42.686489 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.686457    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:28:42.718840 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.718802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgqgg\" (UniqueName: \"kubernetes.io/projected/2dca0af8-0bb4-4c78-b365-5a2be92f4c5c-kube-api-access-vgqgg\") pod \"monitoring-plugin-5d85778b9d-s595m\" (UID: \"2dca0af8-0bb4-4c78-b365-5a2be92f4c5c\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:42.719054 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.718957    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2dca0af8-0bb4-4c78-b365-5a2be92f4c5c-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-s595m\" (UID: \"2dca0af8-0bb4-4c78-b365-5a2be92f4c5c\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:42.820231 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.820188    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgqgg\" (UniqueName: \"kubernetes.io/projected/2dca0af8-0bb4-4c78-b365-5a2be92f4c5c-kube-api-access-vgqgg\") pod \"monitoring-plugin-5d85778b9d-s595m\" (UID: \"2dca0af8-0bb4-4c78-b365-5a2be92f4c5c\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:42.820406 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.820362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2dca0af8-0bb4-4c78-b365-5a2be92f4c5c-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-s595m\" (UID: \"2dca0af8-0bb4-4c78-b365-5a2be92f4c5c\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:42.820496 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:42.820475    2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 15 22:28:42.820551 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:28:42.820542    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dca0af8-0bb4-4c78-b365-5a2be92f4c5c-monitoring-plugin-cert podName:2dca0af8-0bb4-4c78-b365-5a2be92f4c5c nodeName:}" failed. No retries permitted until 2026-09-15 22:28:43.320529054 +0000 UTC m=+63.074153159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/2dca0af8-0bb4-4c78-b365-5a2be92f4c5c-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-s595m" (UID: "2dca0af8-0bb4-4c78-b365-5a2be92f4c5c") : secret "monitoring-plugin-cert" not found
Sep 15 22:28:42.829352 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.829317    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgqgg\" (UniqueName: \"kubernetes.io/projected/2dca0af8-0bb4-4c78-b365-5a2be92f4c5c-kube-api-access-vgqgg\") pod \"monitoring-plugin-5d85778b9d-s595m\" (UID: \"2dca0af8-0bb4-4c78-b365-5a2be92f4c5c\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:42.965746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:42.965714    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7cc447d874-cknv6"]
Sep 15 22:28:43.325037 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:43.324943    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2dca0af8-0bb4-4c78-b365-5a2be92f4c5c-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-s595m\" (UID: \"2dca0af8-0bb4-4c78-b365-5a2be92f4c5c\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:43.327637 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:43.327600    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2dca0af8-0bb4-4c78-b365-5a2be92f4c5c-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-s595m\" (UID: \"2dca0af8-0bb4-4c78-b365-5a2be92f4c5c\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:43.460611 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:43.460579    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fd2ff5_486f_495c_b00a_dc31cc78c6f3.slice/crio-c23e0bc1d0f7d3edc477b04495267f3f00635a800ae93d8e0caa2d563b613322 WatchSource:0}: Error finding container c23e0bc1d0f7d3edc477b04495267f3f00635a800ae93d8e0caa2d563b613322: Status 404 returned error can't find the container with id c23e0bc1d0f7d3edc477b04495267f3f00635a800ae93d8e0caa2d563b613322
Sep 15 22:28:43.580894 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:43.580865    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:43.858501 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:43.858423    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"]
Sep 15 22:28:43.861379 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:43.861339    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dca0af8_0bb4_4c78_b365_5a2be92f4c5c.slice/crio-74918e5ebe8a2f3284017433308582b5ce806248e70bf06024d6e6a2633dfc49 WatchSource:0}: Error finding container 74918e5ebe8a2f3284017433308582b5ce806248e70bf06024d6e6a2633dfc49: Status 404 returned error can't find the container with id 74918e5ebe8a2f3284017433308582b5ce806248e70bf06024d6e6a2633dfc49
Sep 15 22:28:44.028198 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.028164    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-775566df64-gtk8z"]
Sep 15 22:28:44.030528 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.030495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.041150 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.041120    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-775566df64-gtk8z"]
Sep 15 22:28:44.068987 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.068951    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2" event={"ID":"287cae11-8551-4505-aebe-9f1623157a9a","Type":"ContainerStarted","Data":"9158792cc886345ba66642a82feaee3c3bec26cf098b5fb581e0273f388ef71a"}
Sep 15 22:28:44.069168 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.069031    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2" event={"ID":"287cae11-8551-4505-aebe-9f1623157a9a","Type":"ContainerStarted","Data":"5c9356ac7b6030022142bee1c21bc739227ce287599c39d5c0ca6b17ca1a8a84"}
Sep 15 22:28:44.069168 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.069077    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2" event={"ID":"287cae11-8551-4505-aebe-9f1623157a9a","Type":"ContainerStarted","Data":"4cf6497e526ae7b3433c32398c60927f8f1b5845d29bfe9d586ee8058373514a"}
Sep 15 22:28:44.070255 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.070227    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m" event={"ID":"2dca0af8-0bb4-4c78-b365-5a2be92f4c5c","Type":"ContainerStarted","Data":"74918e5ebe8a2f3284017433308582b5ce806248e70bf06024d6e6a2633dfc49"}
Sep 15 22:28:44.071366 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.071343    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7cc447d874-cknv6" event={"ID":"03fd2ff5-486f-495c-b00a-dc31cc78c6f3","Type":"ContainerStarted","Data":"c23e0bc1d0f7d3edc477b04495267f3f00635a800ae93d8e0caa2d563b613322"}
Sep 15 22:28:44.113009 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.112916    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 22:28:44.116267 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.116239    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.119124 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.119076    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 22:28:44.119330 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.119295    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 22:28:44.119549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.119534    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 22:28:44.119646 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.119623    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 22:28:44.119755 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.119737    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5zxhg\""
Sep 15 22:28:44.120770 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.120749    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 22:28:44.120925 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.120866    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 22:28:44.120925 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.120891    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 22:28:44.121045 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.120996    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 22:28:44.121247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.121228    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1b81sag5l8i\""
Sep 15 22:28:44.121247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.121241    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 22:28:44.121369 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.121228    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 22:28:44.121509 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.121491    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 22:28:44.121561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.121524    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 22:28:44.121612 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.121586    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 22:28:44.123033 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.123017    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 22:28:44.131327 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.131294    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-console-config\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.131459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.131343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-service-ca\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.131459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.131441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-oauth-config\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.131545 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.131516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-oauth-serving-cert\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.131545 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.131539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-serving-cert\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.131600 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.131560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-trusted-ca-bundle\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.131600 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.131575    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5txzl\" (UniqueName: \"kubernetes.io/projected/6167c015-8623-4382-8541-997ac12bcf7a-kube-api-access-5txzl\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.135617 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.135577    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 22:28:44.232794 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.232756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233025 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.232821    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-oauth-config\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.233025 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.232852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233025 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.232876    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233025 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.232900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2q9f\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-kube-api-access-q2q9f\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233025 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.232992    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233029    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-config\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233114    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233142    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-oauth-serving-cert\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.233315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-serving-cert\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.233315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233261    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-trusted-ca-bundle\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.233315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233286    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5txzl\" (UniqueName: \"kubernetes.io/projected/6167c015-8623-4382-8541-997ac12bcf7a-kube-api-access-5txzl\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.233315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-console-config\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233450    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-web-config\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233479    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233511    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-service-ca\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233656    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.233857 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233850    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-config-out\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.234385 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.233983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.234385 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.234015    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.234385 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.234083    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-oauth-serving-cert\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.234385 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.234290    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-console-config\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.234893 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.234871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-trusted-ca-bundle\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.236106 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.235910    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-service-ca\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.240435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.240362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-serving-cert\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.241555 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.241533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-oauth-config\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.245503 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.245479    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5txzl\" (UniqueName: \"kubernetes.io/projected/6167c015-8623-4382-8541-997ac12bcf7a-kube-api-access-5txzl\") pod \"console-775566df64-gtk8z\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") " pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.334867 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.334826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.334867 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.334871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.334894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2q9f\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-kube-api-access-q2q9f\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335033    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335129    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-config\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335181    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335226    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-web-config\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335361    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.335431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.336020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.336020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335482    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.336020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-config-out\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.336020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335544    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.336020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.336020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.336020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.335714    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.336991 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.336956    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.337496 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.337469    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.338800 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.338458    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.338910 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.338831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.341146 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.339294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.341146 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.339928    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.341146 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.340770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.341485 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.341464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.341640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.341620    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:44.342363 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.342338    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.343511 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.343459    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-web-config\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.343596 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.343543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-config\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.343776 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.343753    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-config-out\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.344920 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.344162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.344920 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.344165    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.344920 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.344522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.344920 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.344884    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.345295 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.345012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.345692 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.345668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.346400 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.346379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2q9f\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-kube-api-access-q2q9f\") pod \"prometheus-k8s-0\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.428120 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.428049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:44.634037 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.633924    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-775566df64-gtk8z"]
Sep 15 22:28:44.644724 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:44.644672    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6167c015_8623_4382_8541_997ac12bcf7a.slice/crio-24787d6c7bced74236b723652423a9e6736b934e7b4723ac894eb4b2ae165b13 WatchSource:0}: Error finding container 24787d6c7bced74236b723652423a9e6736b934e7b4723ac894eb4b2ae165b13: Status 404 returned error can't find the container with id 24787d6c7bced74236b723652423a9e6736b934e7b4723ac894eb4b2ae165b13
Sep 15 22:28:44.721539 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:44.721427    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 22:28:45.031710 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:45.031677    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-crlld"
Sep 15 22:28:45.080535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:45.080495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775566df64-gtk8z" event={"ID":"6167c015-8623-4382-8541-997ac12bcf7a","Type":"ContainerStarted","Data":"e1b9ddffa6dc2d2d4b69ee6f04063fb855cb56df31094b795b51476a34948019"}
Sep 15 22:28:45.080727 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:45.080560    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775566df64-gtk8z" event={"ID":"6167c015-8623-4382-8541-997ac12bcf7a","Type":"ContainerStarted","Data":"24787d6c7bced74236b723652423a9e6736b934e7b4723ac894eb4b2ae165b13"}
Sep 15 22:28:45.082461 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:45.082425    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerStarted","Data":"32161263f549b7337da926756e9b03a20f337fd622b5104eeb4b1cb0b66922bf"}
Sep 15 22:28:45.106498 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:45.106369    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-775566df64-gtk8z" podStartSLOduration=1.10634587 podStartE2EDuration="1.10634587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:45.105712349 +0000 UTC m=+64.859336478" watchObservedRunningTime="2026-09-15 22:28:45.10634587 +0000 UTC m=+64.859969999"
Sep 15 22:28:46.090445 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.089628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m" event={"ID":"2dca0af8-0bb4-4c78-b365-5a2be92f4c5c","Type":"ContainerStarted","Data":"8bd966d7e4c5743702b8684eeededbe559a8f9bfd63eade779c19ad2b9ca4e5f"}
Sep 15 22:28:46.090445 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.089895    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:46.092958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.092886    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7cc447d874-cknv6" event={"ID":"03fd2ff5-486f-495c-b00a-dc31cc78c6f3","Type":"ContainerStarted","Data":"d114c36889173710de4e59b4c5f7a229533b3a8658b01cd48b8d2cd07e630334"}
Sep 15 22:28:46.096236 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.096211    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m"
Sep 15 22:28:46.096991 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.096962    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2" event={"ID":"287cae11-8551-4505-aebe-9f1623157a9a","Type":"ContainerStarted","Data":"2d98121f460a36d2fce24c7e8f3e7e31a16d9b2e25ae5240e0518875b7e59c89"}
Sep 15 22:28:46.097116 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.097040    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2" event={"ID":"287cae11-8551-4505-aebe-9f1623157a9a","Type":"ContainerStarted","Data":"139a0a5ed0e4dc60d7effd5b7d88ee942f76cf2725f728c1cf386c3cc7d8a1ce"}
Sep 15 22:28:46.097116 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.097082    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2" event={"ID":"287cae11-8551-4505-aebe-9f1623157a9a","Type":"ContainerStarted","Data":"76cbf16d7a0755df4d7826b50d4c432795741a3beccba9cd5f5ccb44a2acbbb2"}
Sep 15 22:28:46.107557 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.107494    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-s595m" podStartSLOduration=2.185838411 podStartE2EDuration="4.107469405s" totalImagesPullingTime="1.921630994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:46.106857722 +0000 UTC m=+65.860481852" watchObservedRunningTime="2026-09-15 22:28:46.107469405 +0000 UTC m=+65.861093534"
Sep 15 22:28:46.163338 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.162525    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2" podStartSLOduration=2.125012608 podStartE2EDuration="6.162502912s" totalImagesPullingTime="4.037490304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:46.161582046 +0000 UTC m=+65.915206176" watchObservedRunningTime="2026-09-15 22:28:46.162502912 +0000 UTC m=+65.916127040"
Sep 15 22:28:46.183431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.183370    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7cc447d874-cknv6" podStartSLOduration=1.863809619 podStartE2EDuration="4.183346971s" totalImagesPullingTime="2.319537352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:46.182976068 +0000 UTC m=+65.936600199" watchObservedRunningTime="2026-09-15 22:28:46.183346971 +0000 UTC m=+65.936971100"
Sep 15 22:28:46.460604 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.460568    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:46.460751 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.460615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:46.463557 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.463533    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 22:28:46.463629 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.463534    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 22:28:46.473180 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.473152    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b02d944-b4ba-440f-8ec2-bec251299069-metrics-certs\") pod \"network-metrics-daemon-tbgzs\" (UID: \"7b02d944-b4ba-440f-8ec2-bec251299069\") " pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:46.473353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.473339    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 22:28:46.483924 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.483898    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z2t8\" (UniqueName: \"kubernetes.io/projected/bae50b7e-fb4b-4da6-b71e-c824b2b6d295-kube-api-access-8z2t8\") pod \"network-check-target-rc44q\" (UID: \"bae50b7e-fb4b-4da6-b71e-c824b2b6d295\") " pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:46.561968 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.561937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:46.564712 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.564689    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 22:28:46.573575 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.573552    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fznt9\""
Sep 15 22:28:46.575618 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.575595    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b32c671-b247-4256-878e-a286c276b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-dssx6\" (UID: \"0b32c671-b247-4256-878e-a286c276b9ca\") " pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:46.578129 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.578104    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dssx6"
Sep 15 22:28:46.580984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.580967    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:46.587138 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.587114    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-29265\""
Sep 15 22:28:46.595284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.595245    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tbgzs"
Sep 15 22:28:46.866493 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.866346    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dssx6"]
Sep 15 22:28:46.868928 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:46.868899    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b32c671_b247_4256_878e_a286c276b9ca.slice/crio-d13a094e10101cb2b4c4af41fc0214bf392c8a7a126c397315d81cf504a8aed5 WatchSource:0}: Error finding container d13a094e10101cb2b4c4af41fc0214bf392c8a7a126c397315d81cf504a8aed5: Status 404 returned error can't find the container with id d13a094e10101cb2b4c4af41fc0214bf392c8a7a126c397315d81cf504a8aed5
Sep 15 22:28:46.875398 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.875375    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rc44q"]
Sep 15 22:28:46.878555 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:46.878525    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae50b7e_fb4b_4da6_b71e_c824b2b6d295.slice/crio-fe28a4603f147e936fb675a1723d99fb3eb6fba5f491242dd0667e7080deed15 WatchSource:0}: Error finding container fe28a4603f147e936fb675a1723d99fb3eb6fba5f491242dd0667e7080deed15: Status 404 returned error can't find the container with id fe28a4603f147e936fb675a1723d99fb3eb6fba5f491242dd0667e7080deed15
Sep 15 22:28:46.898694 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:46.898660    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tbgzs"]
Sep 15 22:28:46.901748 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:28:46.901710    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b02d944_b4ba_440f_8ec2_bec251299069.slice/crio-cb3292864ecc9389f57589f58860854e5382f74e600a0f1f83bd4461f30468e9 WatchSource:0}: Error finding container cb3292864ecc9389f57589f58860854e5382f74e600a0f1f83bd4461f30468e9: Status 404 returned error can't find the container with id cb3292864ecc9389f57589f58860854e5382f74e600a0f1f83bd4461f30468e9
Sep 15 22:28:47.102777 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:47.102740    2578 generic.go:356] "Generic (PLEG): container finished" podID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerID="ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f" exitCode=0
Sep 15 22:28:47.103199 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:47.102790    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerDied","Data":"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"}
Sep 15 22:28:47.104321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:47.104290    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tbgzs" event={"ID":"7b02d944-b4ba-440f-8ec2-bec251299069","Type":"ContainerStarted","Data":"cb3292864ecc9389f57589f58860854e5382f74e600a0f1f83bd4461f30468e9"}
Sep 15 22:28:47.105717 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:47.105689    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rc44q" event={"ID":"bae50b7e-fb4b-4da6-b71e-c824b2b6d295","Type":"ContainerStarted","Data":"fe28a4603f147e936fb675a1723d99fb3eb6fba5f491242dd0667e7080deed15"}
Sep 15 22:28:47.107012 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:47.106980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dssx6" event={"ID":"0b32c671-b247-4256-878e-a286c276b9ca","Type":"ContainerStarted","Data":"d13a094e10101cb2b4c4af41fc0214bf392c8a7a126c397315d81cf504a8aed5"}
Sep 15 22:28:47.107381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:47.107366    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:48.120888 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:48.120564    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8cd6b5b54-rspv2"
Sep 15 22:28:49.124954 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:49.124913    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tbgzs" event={"ID":"7b02d944-b4ba-440f-8ec2-bec251299069","Type":"ContainerStarted","Data":"c87264b81448889ab8b6d241cde360fbdd77bbb6060d8859563d4d30540b2e31"}
Sep 15 22:28:49.125486 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:49.124982    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tbgzs" event={"ID":"7b02d944-b4ba-440f-8ec2-bec251299069","Type":"ContainerStarted","Data":"0a1b1b45ae7be7cc0a48f029bf6af5159cb5a1371ce2af02440a208678325e33"}
Sep 15 22:28:49.144723 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:49.144178    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tbgzs" podStartSLOduration=66.987734394 podStartE2EDuration="1m8.144152836s" totalImagesPullingTime="1.156418442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:49.14370252 +0000 UTC m=+68.897326648" watchObservedRunningTime="2026-09-15 22:28:49.144152836 +0000 UTC m=+68.897776963"
Sep 15 22:28:54.150921 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.150881    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerStarted","Data":"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"}
Sep 15 22:28:54.151426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.150967    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerStarted","Data":"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"}
Sep 15 22:28:54.151426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.151010    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerStarted","Data":"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"}
Sep 15 22:28:54.151426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.151048    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerStarted","Data":"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"}
Sep 15 22:28:54.151426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.151120    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerStarted","Data":"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"}
Sep 15 22:28:54.151426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.151158    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerStarted","Data":"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"}
Sep 15 22:28:54.152495 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.152462    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rc44q" event={"ID":"bae50b7e-fb4b-4da6-b71e-c824b2b6d295","Type":"ContainerStarted","Data":"59cad727f8d0b3b6474032655db7eb6ad8ca8457009969d5aa360d3710a8463c"}
Sep 15 22:28:54.152591 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.152528    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:28:54.153941 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.153907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dssx6" event={"ID":"0b32c671-b247-4256-878e-a286c276b9ca","Type":"ContainerStarted","Data":"2415e07c6e880b75b178abfec9525f0c2bd8e50c46ff9365929b7141d1c99d46"}
Sep 15 22:28:54.185274 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.185219    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.239377693 podStartE2EDuration="10.18519953s" totalImagesPullingTime="7.924192523s" totalInitContainerRuntime="21.629314ms" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:54.182120977 +0000 UTC m=+73.935745103" watchObservedRunningTime="2026-09-15 22:28:54.18519953 +0000 UTC m=+73.938823656"
Sep 15 22:28:54.200478 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.200419    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rc44q" podStartSLOduration=66.83989741 podStartE2EDuration="1m13.200396979s" totalImagesPullingTime="6.360499569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:54.200176423 +0000 UTC m=+73.953800551" watchObservedRunningTime="2026-09-15 22:28:54.200396979 +0000 UTC m=+73.954021107"
Sep 15 22:28:54.216089 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.216024    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dssx6" podStartSLOduration=65.832850813 podStartE2EDuration="1m12.216003084s" totalImagesPullingTime="6.383152271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:54.215773257 +0000 UTC m=+73.969397464" watchObservedRunningTime="2026-09-15 22:28:54.216003084 +0000 UTC m=+73.969627211"
Sep 15 22:28:54.342476 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.342428    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:54.342655 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.342547    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:54.347222 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.347199    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:54.428634 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:54.428559    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:28:55.161757 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:55.161729    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:28:55.213403 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:28:55.213372    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7767f5798b-8nqkg"]
Sep 15 22:29:02.687575 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:02.687430    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:29:02.687575 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:02.687545    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:29:20.232990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.232926    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7767f5798b-8nqkg" podUID="50f82e78-2ca9-47d3-984d-e4f31ab9cf09" containerName="console" containerID="cri-o://28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070" gracePeriod=15
Sep 15 22:29:20.468346 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.468322    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:29:20.569153 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569032    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-oauth-serving-cert\") pod \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") "
Sep 15 22:29:20.569153 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569116    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjt4m\" (UniqueName: \"kubernetes.io/projected/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-kube-api-access-hjt4m\") pod \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") "
Sep 15 22:29:20.569380 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569153    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-config\") pod \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") "
Sep 15 22:29:20.569380 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569178    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-service-ca\") pod \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") "
Sep 15 22:29:20.569380 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569203    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-oauth-config\") pod \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") "
Sep 15 22:29:20.569380 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569227    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-serving-cert\") pod \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") "
Sep 15 22:29:20.569380 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569253    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-trusted-ca-bundle\") pod \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\" (UID: \"50f82e78-2ca9-47d3-984d-e4f31ab9cf09\") "
Sep 15 22:29:20.569627 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569521    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "50f82e78-2ca9-47d3-984d-e4f31ab9cf09" (UID: "50f82e78-2ca9-47d3-984d-e4f31ab9cf09"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:29:20.569698 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569620    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-config" (OuterVolumeSpecName: "console-config") pod "50f82e78-2ca9-47d3-984d-e4f31ab9cf09" (UID: "50f82e78-2ca9-47d3-984d-e4f31ab9cf09"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:29:20.569698 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569673    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-service-ca" (OuterVolumeSpecName: "service-ca") pod "50f82e78-2ca9-47d3-984d-e4f31ab9cf09" (UID: "50f82e78-2ca9-47d3-984d-e4f31ab9cf09"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:29:20.569800 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.569745    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "50f82e78-2ca9-47d3-984d-e4f31ab9cf09" (UID: "50f82e78-2ca9-47d3-984d-e4f31ab9cf09"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:29:20.571518 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.571486    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "50f82e78-2ca9-47d3-984d-e4f31ab9cf09" (UID: "50f82e78-2ca9-47d3-984d-e4f31ab9cf09"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:20.571518 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.571507    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-kube-api-access-hjt4m" (OuterVolumeSpecName: "kube-api-access-hjt4m") pod "50f82e78-2ca9-47d3-984d-e4f31ab9cf09" (UID: "50f82e78-2ca9-47d3-984d-e4f31ab9cf09"). InnerVolumeSpecName "kube-api-access-hjt4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:29:20.571665 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.571538    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "50f82e78-2ca9-47d3-984d-e4f31ab9cf09" (UID: "50f82e78-2ca9-47d3-984d-e4f31ab9cf09"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:20.670031 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.669982    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-oauth-serving-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:29:20.670031 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.670022    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjt4m\" (UniqueName: \"kubernetes.io/projected/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-kube-api-access-hjt4m\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:29:20.670031 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.670038    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-config\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:29:20.670306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.670052    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-service-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:29:20.670306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.670063    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-oauth-config\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:29:20.670306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.670074    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-console-serving-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:29:20.670306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:20.670085    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50f82e78-2ca9-47d3-984d-e4f31ab9cf09-trusted-ca-bundle\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:29:21.257765 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:21.257722    2578 generic.go:356] "Generic (PLEG): container finished" podID="50f82e78-2ca9-47d3-984d-e4f31ab9cf09" containerID="28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070" exitCode=0
Sep 15 22:29:21.258209 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:21.257781    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7767f5798b-8nqkg"
Sep 15 22:29:21.258209 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:21.257813    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7767f5798b-8nqkg" event={"ID":"50f82e78-2ca9-47d3-984d-e4f31ab9cf09","Type":"ContainerDied","Data":"28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070"}
Sep 15 22:29:21.258209 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:21.257869    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7767f5798b-8nqkg" event={"ID":"50f82e78-2ca9-47d3-984d-e4f31ab9cf09","Type":"ContainerDied","Data":"31834d985df2f8181591461759d1ceee47b3beb626ae203b6143bca7aded4b50"}
Sep 15 22:29:21.258209 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:21.257906    2578 scope.go:117] "RemoveContainer" containerID="28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070"
Sep 15 22:29:21.266490 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:21.266467    2578 scope.go:117] "RemoveContainer" containerID="28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070"
Sep 15 22:29:21.266833 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:29:21.266812    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070\": container with ID starting with 28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070 not found: ID does not exist" containerID="28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070"
Sep 15 22:29:21.266886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:21.266844    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070"} err="failed to get container status \"28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070\": rpc error: code = NotFound desc = could not find container \"28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070\": container with ID starting with 28ee217853c07ef8d42293a35d88e30f5c8b3471c1139c7d0094d807bbd1d070 not found: ID does not exist"
Sep 15 22:29:21.276332 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:21.276301    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7767f5798b-8nqkg"]
Sep 15 22:29:21.278697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:21.278671    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7767f5798b-8nqkg"]
Sep 15 22:29:22.692215 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:22.692186    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:29:22.696148 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:22.696113    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7cc447d874-cknv6"
Sep 15 22:29:22.762038 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:22.761996    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f82e78-2ca9-47d3-984d-e4f31ab9cf09" path="/var/lib/kubelet/pods/50f82e78-2ca9-47d3-984d-e4f31ab9cf09/volumes"
Sep 15 22:29:25.160805 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:25.160771    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rc44q"
Sep 15 22:29:44.429213 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:44.429161    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:29:44.448926 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:44.448896    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:29:45.363253 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:29:45.363226    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:02.535079 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.535030    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 22:30:02.535510 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.535456    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="thanos-sidecar" containerID="cri-o://9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e" gracePeriod=600
Sep 15 22:30:02.535510 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.535456    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy" containerID="cri-o://c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110" gracePeriod=600
Sep 15 22:30:02.535639 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.535511    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy-web" containerID="cri-o://ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402" gracePeriod=600
Sep 15 22:30:02.535639 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.535509    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="config-reloader" containerID="cri-o://6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829" gracePeriod=600
Sep 15 22:30:02.535639 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.535438    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="prometheus" containerID="cri-o://79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793" gracePeriod=600
Sep 15 22:30:02.535781 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.535616    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4" gracePeriod=600
Sep 15 22:30:02.789313 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.789232    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:02.830867 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.830825    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.830920    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2q9f\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-kube-api-access-q2q9f\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.830946    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-tls\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.830971    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-db\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831008    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-0\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831050    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-web-config\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831079    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-config-out\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831122    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-grpc-tls\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831148    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-metrics-client-ca\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831171    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-config\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831204    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-serving-certs-ca-bundle\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831231    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-tls-assets\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831269    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831310    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-metrics-client-certs\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831349    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-2\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831377    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-kube-rbac-proxy\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831416    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-thanos-prometheus-http-client-file\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831909 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831441    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-1\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.831909 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.831758    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:02.832410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.832144    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:02.834039 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.833505    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:30:02.834039 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.833943    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:02.834304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.834274    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:02.834366 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.834350    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:02.834889 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.834865    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-trusted-ca-bundle\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.834989 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.834929    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-kubelet-serving-ca-bundle\") pod \"0d753cc5-c736-4e67-a705-acba5d021cd4\" (UID: \"0d753cc5-c736-4e67-a705-acba5d021cd4\") "
Sep 15 22:30:02.835237 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.835217    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.835321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.835242    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.835321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.835257    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-db\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.835321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.835271    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.835321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.835285    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-metrics-client-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.835321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.835300    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.835931 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.835900    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:02.836144 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.836088    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:02.841711 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.841669    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-kube-api-access-q2q9f" (OuterVolumeSpecName: "kube-api-access-q2q9f") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "kube-api-access-q2q9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:30:02.841881 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.841778    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:02.841943 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.841905    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-config" (OuterVolumeSpecName: "config") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:02.842396 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.842327    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:02.842518 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.842490    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:02.842595 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.842550    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:30:02.842746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.842721    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-config-out" (OuterVolumeSpecName: "config-out") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:30:02.843199 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.843168    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:02.843317 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.843217    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:02.843933 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.843903    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:02.844572 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.844548    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:02.854786 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.854728    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-web-config" (OuterVolumeSpecName: "web-config") pod "0d753cc5-c736-4e67-a705-acba5d021cd4" (UID: "0d753cc5-c736-4e67-a705-acba5d021cd4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:02.936488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936446    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-web-config\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936475    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d753cc5-c736-4e67-a705-acba5d021cd4-config-out\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936485    2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-grpc-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936494    2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-config\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936502    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-tls-assets\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936510    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936519    2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-metrics-client-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936528    2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-kube-rbac-proxy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936537    2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936547    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936555    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d753cc5-c736-4e67-a705-acba5d021cd4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936563    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936571    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2q9f\" (UniqueName: \"kubernetes.io/projected/0d753cc5-c736-4e67-a705-acba5d021cd4-kube-api-access-q2q9f\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:02.936810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:02.936581    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d753cc5-c736-4e67-a705-acba5d021cd4-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:03.416191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416156    2578 generic.go:356] "Generic (PLEG): container finished" podID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerID="2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4" exitCode=0
Sep 15 22:30:03.416191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416182    2578 generic.go:356] "Generic (PLEG): container finished" podID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerID="c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110" exitCode=0
Sep 15 22:30:03.416191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416191    2578 generic.go:356] "Generic (PLEG): container finished" podID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerID="ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402" exitCode=0
Sep 15 22:30:03.416191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416197    2578 generic.go:356] "Generic (PLEG): container finished" podID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerID="9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e" exitCode=0
Sep 15 22:30:03.416191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416203    2578 generic.go:356] "Generic (PLEG): container finished" podID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerID="6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829" exitCode=0
Sep 15 22:30:03.416477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416208    2578 generic.go:356] "Generic (PLEG): container finished" podID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerID="79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793" exitCode=0
Sep 15 22:30:03.416477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416232    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerDied","Data":"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"}
Sep 15 22:30:03.416477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416245    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.416477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerDied","Data":"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"}
Sep 15 22:30:03.416477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416355    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerDied","Data":"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"}
Sep 15 22:30:03.416477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerDied","Data":"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"}
Sep 15 22:30:03.416477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416391    2578 scope.go:117] "RemoveContainer" containerID="2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"
Sep 15 22:30:03.416477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerDied","Data":"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"}
Sep 15 22:30:03.416477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416460    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerDied","Data":"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"}
Sep 15 22:30:03.416825 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.416487    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d753cc5-c736-4e67-a705-acba5d021cd4","Type":"ContainerDied","Data":"32161263f549b7337da926756e9b03a20f337fd622b5104eeb4b1cb0b66922bf"}
Sep 15 22:30:03.424573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.424551    2578 scope.go:117] "RemoveContainer" containerID="c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"
Sep 15 22:30:03.431762 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.431742    2578 scope.go:117] "RemoveContainer" containerID="ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"
Sep 15 22:30:03.438705 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.438682    2578 scope.go:117] "RemoveContainer" containerID="9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"
Sep 15 22:30:03.443128 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.443085    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 22:30:03.447024 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.446997    2578 scope.go:117] "RemoveContainer" containerID="6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"
Sep 15 22:30:03.448015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.447994    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 22:30:03.454467 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.454447    2578 scope.go:117] "RemoveContainer" containerID="79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"
Sep 15 22:30:03.462167 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.462145    2578 scope.go:117] "RemoveContainer" containerID="ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"
Sep 15 22:30:03.469482 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.469461    2578 scope.go:117] "RemoveContainer" containerID="2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"
Sep 15 22:30:03.469798 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:30:03.469778    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": container with ID starting with 2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4 not found: ID does not exist" containerID="2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"
Sep 15 22:30:03.469864 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.469809    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"} err="failed to get container status \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": rpc error: code = NotFound desc = could not find container \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": container with ID starting with 2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4 not found: ID does not exist"
Sep 15 22:30:03.469864 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.469831    2578 scope.go:117] "RemoveContainer" containerID="c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"
Sep 15 22:30:03.470129 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:30:03.470108    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": container with ID starting with c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110 not found: ID does not exist" containerID="c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"
Sep 15 22:30:03.470199 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.470139    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"} err="failed to get container status \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": rpc error: code = NotFound desc = could not find container \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": container with ID starting with c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110 not found: ID does not exist"
Sep 15 22:30:03.470199 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.470162    2578 scope.go:117] "RemoveContainer" containerID="ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"
Sep 15 22:30:03.470402 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:30:03.470385    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": container with ID starting with ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402 not found: ID does not exist" containerID="ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"
Sep 15 22:30:03.470439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.470405    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"} err="failed to get container status \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": rpc error: code = NotFound desc = could not find container \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": container with ID starting with ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402 not found: ID does not exist"
Sep 15 22:30:03.470439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.470421    2578 scope.go:117] "RemoveContainer" containerID="9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"
Sep 15 22:30:03.470653 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:30:03.470636    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": container with ID starting with 9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e not found: ID does not exist" containerID="9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"
Sep 15 22:30:03.470695 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.470660    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"} err="failed to get container status \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": rpc error: code = NotFound desc = could not find container \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": container with ID starting with 9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e not found: ID does not exist"
Sep 15 22:30:03.470695 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.470674    2578 scope.go:117] "RemoveContainer" containerID="6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"
Sep 15 22:30:03.470872 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:30:03.470858    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": container with ID starting with 6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829 not found: ID does not exist" containerID="6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"
Sep 15 22:30:03.470911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.470875    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"} err="failed to get container status \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": rpc error: code = NotFound desc = could not find container \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": container with ID starting with 6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829 not found: ID does not exist"
Sep 15 22:30:03.470911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.470886    2578 scope.go:117] "RemoveContainer" containerID="79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"
Sep 15 22:30:03.471117 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:30:03.471079    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": container with ID starting with 79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793 not found: ID does not exist" containerID="79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"
Sep 15 22:30:03.471189 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.471153    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"} err="failed to get container status \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": rpc error: code = NotFound desc = could not find container \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": container with ID starting with 79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793 not found: ID does not exist"
Sep 15 22:30:03.471189 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.471177    2578 scope.go:117] "RemoveContainer" containerID="ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"
Sep 15 22:30:03.471449 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:30:03.471432    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": container with ID starting with ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f not found: ID does not exist" containerID="ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"
Sep 15 22:30:03.471493 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.471454    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"} err="failed to get container status \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": rpc error: code = NotFound desc = could not find container \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": container with ID starting with ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f not found: ID does not exist"
Sep 15 22:30:03.471493 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.471467    2578 scope.go:117] "RemoveContainer" containerID="2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"
Sep 15 22:30:03.471722 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.471702    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"} err="failed to get container status \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": rpc error: code = NotFound desc = could not find container \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": container with ID starting with 2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4 not found: ID does not exist"
Sep 15 22:30:03.471777 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.471721    2578 scope.go:117] "RemoveContainer" containerID="c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"
Sep 15 22:30:03.471961 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.471944    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"} err="failed to get container status \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": rpc error: code = NotFound desc = could not find container \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": container with ID starting with c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110 not found: ID does not exist"
Sep 15 22:30:03.472002 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.471961    2578 scope.go:117] "RemoveContainer" containerID="ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"
Sep 15 22:30:03.472214 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.472194    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"} err="failed to get container status \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": rpc error: code = NotFound desc = could not find container \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": container with ID starting with ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402 not found: ID does not exist"
Sep 15 22:30:03.472214 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.472212    2578 scope.go:117] "RemoveContainer" containerID="9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"
Sep 15 22:30:03.472425 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.472406    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"} err="failed to get container status \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": rpc error: code = NotFound desc = could not find container \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": container with ID starting with 9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e not found: ID does not exist"
Sep 15 22:30:03.472466 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.472426    2578 scope.go:117] "RemoveContainer" containerID="6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"
Sep 15 22:30:03.472662 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.472644    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"} err="failed to get container status \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": rpc error: code = NotFound desc = could not find container \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": container with ID starting with 6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829 not found: ID does not exist"
Sep 15 22:30:03.472662 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.472662    2578 scope.go:117] "RemoveContainer" containerID="79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"
Sep 15 22:30:03.472886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.472871    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"} err="failed to get container status \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": rpc error: code = NotFound desc = could not find container \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": container with ID starting with 79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793 not found: ID does not exist"
Sep 15 22:30:03.472927 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.472887    2578 scope.go:117] "RemoveContainer" containerID="ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"
Sep 15 22:30:03.473115 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473082    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"} err="failed to get container status \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": rpc error: code = NotFound desc = could not find container \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": container with ID starting with ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f not found: ID does not exist"
Sep 15 22:30:03.473164 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473116    2578 scope.go:117] "RemoveContainer" containerID="2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"
Sep 15 22:30:03.473317 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473302    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"} err="failed to get container status \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": rpc error: code = NotFound desc = could not find container \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": container with ID starting with 2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4 not found: ID does not exist"
Sep 15 22:30:03.473357 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473317    2578 scope.go:117] "RemoveContainer" containerID="c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"
Sep 15 22:30:03.473535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473518    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"} err="failed to get container status \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": rpc error: code = NotFound desc = could not find container \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": container with ID starting with c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110 not found: ID does not exist"
Sep 15 22:30:03.473572 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473535    2578 scope.go:117] "RemoveContainer" containerID="ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"
Sep 15 22:30:03.473739 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473721    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"} err="failed to get container status \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": rpc error: code = NotFound desc = could not find container \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": container with ID starting with ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402 not found: ID does not exist"
Sep 15 22:30:03.473786 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473740    2578 scope.go:117] "RemoveContainer" containerID="9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"
Sep 15 22:30:03.473977 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473959    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"} err="failed to get container status \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": rpc error: code = NotFound desc = could not find container \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": container with ID starting with 9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e not found: ID does not exist"
Sep 15 22:30:03.473977 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.473976    2578 scope.go:117] "RemoveContainer" containerID="6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"
Sep 15 22:30:03.474247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.474209    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"} err="failed to get container status \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": rpc error: code = NotFound desc = could not find container \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": container with ID starting with 6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829 not found: ID does not exist"
Sep 15 22:30:03.474247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.474230    2578 scope.go:117] "RemoveContainer" containerID="79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"
Sep 15 22:30:03.474542 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.474504    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"} err="failed to get container status \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": rpc error: code = NotFound desc = could not find container \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": container with ID starting with 79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793 not found: ID does not exist"
Sep 15 22:30:03.474642 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.474547    2578 scope.go:117] "RemoveContainer" containerID="ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"
Sep 15 22:30:03.474867 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.474826    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"} err="failed to get container status \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": rpc error: code = NotFound desc = could not find container \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": container with ID starting with ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f not found: ID does not exist"
Sep 15 22:30:03.474974 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.474870    2578 scope.go:117] "RemoveContainer" containerID="2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"
Sep 15 22:30:03.475238 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.475215    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"} err="failed to get container status \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": rpc error: code = NotFound desc = could not find container \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": container with ID starting with 2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4 not found: ID does not exist"
Sep 15 22:30:03.475238 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.475237    2578 scope.go:117] "RemoveContainer" containerID="c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"
Sep 15 22:30:03.475496 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.475473    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"} err="failed to get container status \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": rpc error: code = NotFound desc = could not find container \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": container with ID starting with c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110 not found: ID does not exist"
Sep 15 22:30:03.475496 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.475495    2578 scope.go:117] "RemoveContainer" containerID="ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"
Sep 15 22:30:03.475759 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.475735    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"} err="failed to get container status \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": rpc error: code = NotFound desc = could not find container \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": container with ID starting with ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402 not found: ID does not exist"
Sep 15 22:30:03.475759 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.475758    2578 scope.go:117] "RemoveContainer" containerID="9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"
Sep 15 22:30:03.476024 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476004    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"} err="failed to get container status \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": rpc error: code = NotFound desc = could not find container \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": container with ID starting with 9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e not found: ID does not exist"
Sep 15 22:30:03.476110 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476026    2578 scope.go:117] "RemoveContainer" containerID="6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"
Sep 15 22:30:03.476173 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476149    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 22:30:03.476282 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476264    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"} err="failed to get container status \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": rpc error: code = NotFound desc = could not find container \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": container with ID starting with 6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829 not found: ID does not exist"
Sep 15 22:30:03.476340 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476283    2578 scope.go:117] "RemoveContainer" containerID="79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"
Sep 15 22:30:03.476403 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476390    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f82e78-2ca9-47d3-984d-e4f31ab9cf09" containerName="console"
Sep 15 22:30:03.476453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476407    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f82e78-2ca9-47d3-984d-e4f31ab9cf09" containerName="console"
Sep 15 22:30:03.476453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476424    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy"
Sep 15 22:30:03.476453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476435    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy"
Sep 15 22:30:03.476453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476445    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="init-config-reloader"
Sep 15 22:30:03.476453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476453    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="init-config-reloader"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476459    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy-web"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476464    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy-web"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476474    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="prometheus"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476479    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="prometheus"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476490    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="config-reloader"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476498    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="config-reloader"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476505    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy-thanos"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476510    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy-thanos"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476513    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"} err="failed to get container status \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": rpc error: code = NotFound desc = could not find container \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": container with ID starting with 79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793 not found: ID does not exist"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476536    2578 scope.go:117] "RemoveContainer" containerID="ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476521    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="thanos-sidecar"
Sep 15 22:30:03.476619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476586    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="thanos-sidecar"
Sep 15 22:30:03.476984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476681    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="config-reloader"
Sep 15 22:30:03.476984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476692    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="thanos-sidecar"
Sep 15 22:30:03.476984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476700    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy"
Sep 15 22:30:03.476984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476707    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="prometheus"
Sep 15 22:30:03.476984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476711    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy-web"
Sep 15 22:30:03.476984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476717    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" containerName="kube-rbac-proxy-thanos"
Sep 15 22:30:03.476984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476723    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f82e78-2ca9-47d3-984d-e4f31ab9cf09" containerName="console"
Sep 15 22:30:03.476984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476732    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"} err="failed to get container status \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": rpc error: code = NotFound desc = could not find container \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": container with ID starting with ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f not found: ID does not exist"
Sep 15 22:30:03.476984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476747    2578 scope.go:117] "RemoveContainer" containerID="2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"
Sep 15 22:30:03.477281 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.476993    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"} err="failed to get container status \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": rpc error: code = NotFound desc = could not find container \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": container with ID starting with 2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4 not found: ID does not exist"
Sep 15 22:30:03.477281 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.477005    2578 scope.go:117] "RemoveContainer" containerID="c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"
Sep 15 22:30:03.477281 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.477252    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"} err="failed to get container status \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": rpc error: code = NotFound desc = could not find container \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": container with ID starting with c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110 not found: ID does not exist"
Sep 15 22:30:03.477281 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.477270    2578 scope.go:117] "RemoveContainer" containerID="ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"
Sep 15 22:30:03.477526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.477509    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"} err="failed to get container status \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": rpc error: code = NotFound desc = could not find container \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": container with ID starting with ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402 not found: ID does not exist"
Sep 15 22:30:03.477591 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.477527    2578 scope.go:117] "RemoveContainer" containerID="9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"
Sep 15 22:30:03.477743 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.477724    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"} err="failed to get container status \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": rpc error: code = NotFound desc = could not find container \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": container with ID starting with 9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e not found: ID does not exist"
Sep 15 22:30:03.477743 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.477743    2578 scope.go:117] "RemoveContainer" containerID="6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"
Sep 15 22:30:03.477954 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.477932    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"} err="failed to get container status \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": rpc error: code = NotFound desc = could not find container \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": container with ID starting with 6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829 not found: ID does not exist"
Sep 15 22:30:03.478005 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.477957    2578 scope.go:117] "RemoveContainer" containerID="79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"
Sep 15 22:30:03.478198 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.478176    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"} err="failed to get container status \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": rpc error: code = NotFound desc = could not find container \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": container with ID starting with 79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793 not found: ID does not exist"
Sep 15 22:30:03.478198 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.478195    2578 scope.go:117] "RemoveContainer" containerID="ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"
Sep 15 22:30:03.478429 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.478412    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"} err="failed to get container status \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": rpc error: code = NotFound desc = could not find container \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": container with ID starting with ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f not found: ID does not exist"
Sep 15 22:30:03.478429 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.478428    2578 scope.go:117] "RemoveContainer" containerID="2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"
Sep 15 22:30:03.478640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.478621    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4"} err="failed to get container status \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": rpc error: code = NotFound desc = could not find container \"2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4\": container with ID starting with 2b2b832a87174caea5c5578df45dfbce0e9dedf6fece6cad98e9bf0e8f6004a4 not found: ID does not exist"
Sep 15 22:30:03.478712 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.478641    2578 scope.go:117] "RemoveContainer" containerID="c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"
Sep 15 22:30:03.478893 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.478876    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110"} err="failed to get container status \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": rpc error: code = NotFound desc = could not find container \"c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110\": container with ID starting with c47685d666c0be5550562bd1a539259ff169285c41be8ac6a21cd3688ed06110 not found: ID does not exist"
Sep 15 22:30:03.478933 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.478893    2578 scope.go:117] "RemoveContainer" containerID="ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"
Sep 15 22:30:03.479076 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.479058    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402"} err="failed to get container status \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": rpc error: code = NotFound desc = could not find container \"ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402\": container with ID starting with ceeb5c10d101db4ae88991754b076e0b88af75f3cb5c11ece87cc66175563402 not found: ID does not exist"
Sep 15 22:30:03.479156 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.479076    2578 scope.go:117] "RemoveContainer" containerID="9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"
Sep 15 22:30:03.479331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.479312    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e"} err="failed to get container status \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": rpc error: code = NotFound desc = could not find container \"9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e\": container with ID starting with 9332c02afb9160a22e76576e770d8688094a13b99c5cdd7d7cbc7ef43e2b661e not found: ID does not exist"
Sep 15 22:30:03.479331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.479330    2578 scope.go:117] "RemoveContainer" containerID="6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"
Sep 15 22:30:03.479554 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.479533    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829"} err="failed to get container status \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": rpc error: code = NotFound desc = could not find container \"6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829\": container with ID starting with 6cb54d132bbb49e3b73a3ef7c77cc4bac181d7306f76a544be24338534968829 not found: ID does not exist"
Sep 15 22:30:03.479620 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.479555    2578 scope.go:117] "RemoveContainer" containerID="79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"
Sep 15 22:30:03.479792 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.479775    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793"} err="failed to get container status \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": rpc error: code = NotFound desc = could not find container \"79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793\": container with ID starting with 79bdd3c2e2ec0de0ef2c79fbed0e5edb42e7940e94b3d5e22e6d20fee858c793 not found: ID does not exist"
Sep 15 22:30:03.479836 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.479792    2578 scope.go:117] "RemoveContainer" containerID="ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"
Sep 15 22:30:03.480019 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.480002    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f"} err="failed to get container status \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": rpc error: code = NotFound desc = could not find container \"ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f\": container with ID starting with ff0cae7fafe41a739e10130aa9c1a861065cd8543192d1aa630b1848b861950f not found: ID does not exist"
Sep 15 22:30:03.480476 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.480461    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.483227 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.483209    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 22:30:03.484058 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.484027    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 22:30:03.484192 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.484110    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 22:30:03.484431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.484413    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 22:30:03.484664 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.484647    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5zxhg\""
Sep 15 22:30:03.484769 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.484697    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 22:30:03.485546 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.485528    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 22:30:03.485768 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.485752    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 22:30:03.485902 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.485826    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 22:30:03.486001 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.485942    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 22:30:03.486001 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.485983    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 22:30:03.486001 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.485986    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 22:30:03.486384 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.486364    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1b81sag5l8i\""
Sep 15 22:30:03.487788 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.487770    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 22:30:03.489489 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.489468    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 22:30:03.492599 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.492581    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 22:30:03.502264 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.502240    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 22:30:03.541683 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541644    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-config\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.541683 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541700    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541723    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541821    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9brt\" (UniqueName: \"kubernetes.io/projected/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-kube-api-access-x9brt\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541855    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541902    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-config-out\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541947    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.541988    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.542016    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.542035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.542056    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.542085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.542152    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.542168    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542671 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.542188    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-web-config\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.542671 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.542204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643081 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643081 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643079    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643366 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643366 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643278    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643366 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643366 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643367    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-web-config\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643474    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-config\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643554    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9brt\" (UniqueName: \"kubernetes.io/projected/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-kube-api-access-x9brt\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643724    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-config-out\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643750    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643796    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.643871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.644329 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.643961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.646014 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.644552    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.646014 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.645284    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.646014 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.645308    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.646014 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.645877    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.646343 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.646308    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.646776 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.646654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-config\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.647420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.646775    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.647420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.647028    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.647420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.647053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.647420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.647396    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.647658 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.647620    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-config-out\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.648580 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.648548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-web-config\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.649313 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.649280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.649415 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.649365    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.649496 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.649477    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.649580 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.649563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.653558 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.653536    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9brt\" (UniqueName: \"kubernetes.io/projected/fecd4a63-3a50-4e24-88c0-9b278b5e8d30-kube-api-access-x9brt\") pod \"prometheus-k8s-0\" (UID: \"fecd4a63-3a50-4e24-88c0-9b278b5e8d30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:03.791289 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:03.791245    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:04.074744 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:04.074719    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 22:30:04.076343 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:30:04.076307    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfecd4a63_3a50_4e24_88c0_9b278b5e8d30.slice/crio-f7541b806ef9dd59686d3b8444c57f81aec2e01f40513c1ee640cae379546a52 WatchSource:0}: Error finding container f7541b806ef9dd59686d3b8444c57f81aec2e01f40513c1ee640cae379546a52: Status 404 returned error can't find the container with id f7541b806ef9dd59686d3b8444c57f81aec2e01f40513c1ee640cae379546a52
Sep 15 22:30:04.426685 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:04.426566    2578 generic.go:356] "Generic (PLEG): container finished" podID="fecd4a63-3a50-4e24-88c0-9b278b5e8d30" containerID="cb0039c676ae809bdbf6b59cfb2f9434fd931870c5643b54044b9fb9022f089b" exitCode=0
Sep 15 22:30:04.426685 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:04.426636    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fecd4a63-3a50-4e24-88c0-9b278b5e8d30","Type":"ContainerDied","Data":"cb0039c676ae809bdbf6b59cfb2f9434fd931870c5643b54044b9fb9022f089b"}
Sep 15 22:30:04.426903 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:04.426793    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fecd4a63-3a50-4e24-88c0-9b278b5e8d30","Type":"ContainerStarted","Data":"f7541b806ef9dd59686d3b8444c57f81aec2e01f40513c1ee640cae379546a52"}
Sep 15 22:30:04.763311 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:04.762576    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d753cc5-c736-4e67-a705-acba5d021cd4" path="/var/lib/kubelet/pods/0d753cc5-c736-4e67-a705-acba5d021cd4/volumes"
Sep 15 22:30:05.436238 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:05.436205    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fecd4a63-3a50-4e24-88c0-9b278b5e8d30","Type":"ContainerStarted","Data":"4e00cf4dc2215115208e38e1acf08e026e5e96005a6c1f774b5c60d81f43f6d3"}
Sep 15 22:30:05.436409 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:05.436267    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fecd4a63-3a50-4e24-88c0-9b278b5e8d30","Type":"ContainerStarted","Data":"9a0a9c207056c5f337f7bedff63ee3c8174d6e97f2357fabb624bbe547848693"}
Sep 15 22:30:05.436409 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:05.436294    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fecd4a63-3a50-4e24-88c0-9b278b5e8d30","Type":"ContainerStarted","Data":"170de68ea45f6b78b2da645e85397cc9de5b850530f79fe616961e49c4a79806"}
Sep 15 22:30:05.436409 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:05.436322    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fecd4a63-3a50-4e24-88c0-9b278b5e8d30","Type":"ContainerStarted","Data":"5b7af03c28cd0b6bd231782c3e29aa90dbc1b16224a653616195cf8b39620f38"}
Sep 15 22:30:05.436409 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:05.436345    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fecd4a63-3a50-4e24-88c0-9b278b5e8d30","Type":"ContainerStarted","Data":"1c6afbba995045db5c1742e3c4548e4448bced66c7d74b9412ab62d693df31ae"}
Sep 15 22:30:05.436409 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:05.436367    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fecd4a63-3a50-4e24-88c0-9b278b5e8d30","Type":"ContainerStarted","Data":"afb2a5f9551c1916a2e403743ef39806238c8ced64e7648af30d427d30aa8ee8"}
Sep 15 22:30:05.468247 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:05.468171    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.44875297 podStartE2EDuration="2.468147183s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.394213ms" isStatefulPod=false podCreationTimestamp="2026-09-15 22:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:30:05.466713738 +0000 UTC m=+145.220337865" watchObservedRunningTime="2026-09-15 22:30:05.468147183 +0000 UTC m=+145.221771311"
Sep 15 22:30:08.792025 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:08.791987    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:30:17.371899 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:17.371860    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-775566df64-gtk8z"]
Sep 15 22:30:42.397259 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.397215    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-775566df64-gtk8z" podUID="6167c015-8623-4382-8541-997ac12bcf7a" containerName="console" containerID="cri-o://e1b9ddffa6dc2d2d4b69ee6f04063fb855cb56df31094b795b51476a34948019" gracePeriod=15
Sep 15 22:30:42.585257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.585208    2578 generic.go:356] "Generic (PLEG): container finished" podID="6167c015-8623-4382-8541-997ac12bcf7a" containerID="e1b9ddffa6dc2d2d4b69ee6f04063fb855cb56df31094b795b51476a34948019" exitCode=0
Sep 15 22:30:42.585433 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.585293    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775566df64-gtk8z" event={"ID":"6167c015-8623-4382-8541-997ac12bcf7a","Type":"ContainerDied","Data":"e1b9ddffa6dc2d2d4b69ee6f04063fb855cb56df31094b795b51476a34948019"}
Sep 15 22:30:42.635456 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.635432    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:30:42.770246 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770213    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5txzl\" (UniqueName: \"kubernetes.io/projected/6167c015-8623-4382-8541-997ac12bcf7a-kube-api-access-5txzl\") pod \"6167c015-8623-4382-8541-997ac12bcf7a\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") "
Sep 15 22:30:42.770246 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770257    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-trusted-ca-bundle\") pod \"6167c015-8623-4382-8541-997ac12bcf7a\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") "
Sep 15 22:30:42.770520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770296    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-oauth-config\") pod \"6167c015-8623-4382-8541-997ac12bcf7a\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") "
Sep 15 22:30:42.770520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770315    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-serving-cert\") pod \"6167c015-8623-4382-8541-997ac12bcf7a\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") "
Sep 15 22:30:42.770520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770340    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-oauth-serving-cert\") pod \"6167c015-8623-4382-8541-997ac12bcf7a\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") "
Sep 15 22:30:42.770520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770374    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-console-config\") pod \"6167c015-8623-4382-8541-997ac12bcf7a\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") "
Sep 15 22:30:42.770520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770413    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-service-ca\") pod \"6167c015-8623-4382-8541-997ac12bcf7a\" (UID: \"6167c015-8623-4382-8541-997ac12bcf7a\") "
Sep 15 22:30:42.770868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770834    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6167c015-8623-4382-8541-997ac12bcf7a" (UID: "6167c015-8623-4382-8541-997ac12bcf7a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:42.770868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770857    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6167c015-8623-4382-8541-997ac12bcf7a" (UID: "6167c015-8623-4382-8541-997ac12bcf7a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:42.770868 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770843    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-console-config" (OuterVolumeSpecName: "console-config") pod "6167c015-8623-4382-8541-997ac12bcf7a" (UID: "6167c015-8623-4382-8541-997ac12bcf7a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:42.771121 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.770862    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-service-ca" (OuterVolumeSpecName: "service-ca") pod "6167c015-8623-4382-8541-997ac12bcf7a" (UID: "6167c015-8623-4382-8541-997ac12bcf7a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:30:42.772617 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.772583    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6167c015-8623-4382-8541-997ac12bcf7a" (UID: "6167c015-8623-4382-8541-997ac12bcf7a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:42.772719 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.772631    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6167c015-8623-4382-8541-997ac12bcf7a-kube-api-access-5txzl" (OuterVolumeSpecName: "kube-api-access-5txzl") pod "6167c015-8623-4382-8541-997ac12bcf7a" (UID: "6167c015-8623-4382-8541-997ac12bcf7a"). InnerVolumeSpecName "kube-api-access-5txzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:30:42.772719 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.772645    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6167c015-8623-4382-8541-997ac12bcf7a" (UID: "6167c015-8623-4382-8541-997ac12bcf7a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:30:42.871111 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.871067    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-service-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:42.871111 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.871117    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5txzl\" (UniqueName: \"kubernetes.io/projected/6167c015-8623-4382-8541-997ac12bcf7a-kube-api-access-5txzl\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:42.871358 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.871132    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-trusted-ca-bundle\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:42.871358 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.871147    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-oauth-config\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:42.871358 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.871162    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6167c015-8623-4382-8541-997ac12bcf7a-console-serving-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:42.871358 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.871175    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-oauth-serving-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:42.871358 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:42.871188    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6167c015-8623-4382-8541-997ac12bcf7a-console-config\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:30:43.590035 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:43.590009    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775566df64-gtk8z"
Sep 15 22:30:43.590478 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:43.590008    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775566df64-gtk8z" event={"ID":"6167c015-8623-4382-8541-997ac12bcf7a","Type":"ContainerDied","Data":"24787d6c7bced74236b723652423a9e6736b934e7b4723ac894eb4b2ae165b13"}
Sep 15 22:30:43.590478 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:43.590153    2578 scope.go:117] "RemoveContainer" containerID="e1b9ddffa6dc2d2d4b69ee6f04063fb855cb56df31094b795b51476a34948019"
Sep 15 22:30:43.611014 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:43.610983    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-775566df64-gtk8z"]
Sep 15 22:30:43.614870 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:43.614845    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-775566df64-gtk8z"]
Sep 15 22:30:44.761969 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:30:44.761932    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6167c015-8623-4382-8541-997ac12bcf7a" path="/var/lib/kubelet/pods/6167c015-8623-4382-8541-997ac12bcf7a/volumes"
Sep 15 22:31:03.792416 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:31:03.792364    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:31:03.807866 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:31:03.807838    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:31:04.689084 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:31:04.689044    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 22:32:40.645211 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:32:40.645181    2578 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 22:33:03.270479 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.270442    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"]
Sep 15 22:33:03.271869 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.270708    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6167c015-8623-4382-8541-997ac12bcf7a" containerName="console"
Sep 15 22:33:03.271869 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.270718    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6167c015-8623-4382-8541-997ac12bcf7a" containerName="console"
Sep 15 22:33:03.271869 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.270767    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6167c015-8623-4382-8541-997ac12bcf7a" containerName="console"
Sep 15 22:33:03.272564 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.272548    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"
Sep 15 22:33:03.275636 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.275612    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-q5j8j\""
Sep 15 22:33:03.275764 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.275652    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 15 22:33:03.276879 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.276861    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 15 22:33:03.283245 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.283221    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"]
Sep 15 22:33:03.354331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.354291    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cb72563-5068-4515-98cd-395d5593ea19-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n64c7\" (UID: \"8cb72563-5068-4515-98cd-395d5593ea19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"
Sep 15 22:33:03.354331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.354337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7rrd\" (UniqueName: \"kubernetes.io/projected/8cb72563-5068-4515-98cd-395d5593ea19-kube-api-access-m7rrd\") pod \"openshift-lws-operator-fd8ccff4c-n64c7\" (UID: \"8cb72563-5068-4515-98cd-395d5593ea19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"
Sep 15 22:33:03.455137 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.455084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cb72563-5068-4515-98cd-395d5593ea19-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n64c7\" (UID: \"8cb72563-5068-4515-98cd-395d5593ea19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"
Sep 15 22:33:03.455324 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.455150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7rrd\" (UniqueName: \"kubernetes.io/projected/8cb72563-5068-4515-98cd-395d5593ea19-kube-api-access-m7rrd\") pod \"openshift-lws-operator-fd8ccff4c-n64c7\" (UID: \"8cb72563-5068-4515-98cd-395d5593ea19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"
Sep 15 22:33:03.455468 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.455445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cb72563-5068-4515-98cd-395d5593ea19-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n64c7\" (UID: \"8cb72563-5068-4515-98cd-395d5593ea19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"
Sep 15 22:33:03.466270 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.466233    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7rrd\" (UniqueName: \"kubernetes.io/projected/8cb72563-5068-4515-98cd-395d5593ea19-kube-api-access-m7rrd\") pod \"openshift-lws-operator-fd8ccff4c-n64c7\" (UID: \"8cb72563-5068-4515-98cd-395d5593ea19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"
Sep 15 22:33:03.598781 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.598685    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"
Sep 15 22:33:03.870732 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.870675    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7"]
Sep 15 22:33:03.873483 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:33:03.873444    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cb72563_5068_4515_98cd_395d5593ea19.slice/crio-5deb804109a5d2aa5f12af4ca5f9270b4d27be5bfb709acc5cabdffd8370b698 WatchSource:0}: Error finding container 5deb804109a5d2aa5f12af4ca5f9270b4d27be5bfb709acc5cabdffd8370b698: Status 404 returned error can't find the container with id 5deb804109a5d2aa5f12af4ca5f9270b4d27be5bfb709acc5cabdffd8370b698
Sep 15 22:33:03.874972 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:03.874954    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:33:04.095363 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:04.095332    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7" event={"ID":"8cb72563-5068-4515-98cd-395d5593ea19","Type":"ContainerStarted","Data":"5deb804109a5d2aa5f12af4ca5f9270b4d27be5bfb709acc5cabdffd8370b698"}
Sep 15 22:33:07.110023 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:07.109977    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7" event={"ID":"8cb72563-5068-4515-98cd-395d5593ea19","Type":"ContainerStarted","Data":"362a70d2e2e959e6669fa8225f89627d604636c6d37e43a615dddac8de66c977"}
Sep 15 22:33:07.132015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:07.131962    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n64c7" podStartSLOduration=1.5819413390000001 podStartE2EDuration="4.131944955s" totalImagesPullingTime="2.550003616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:33:07.129908059 +0000 UTC m=+326.883532185" watchObservedRunningTime="2026-09-15 22:33:07.131944955 +0000 UTC m=+326.885569083"
Sep 15 22:33:23.875106 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:23.875071    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"]
Sep 15 22:33:23.878083 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:23.878066    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:23.881797 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:23.881770    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 22:33:23.881941 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:23.881872    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 22:33:23.882081 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:23.882061    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 22:33:23.882214 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:23.882116    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hh8gz\""
Sep 15 22:33:23.882463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:23.882447    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 22:33:23.898832 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:23.898804    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"]
Sep 15 22:33:24.020151 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.020078    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/45abfd8e-876c-49f2-965e-fc83d7547be3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.020324 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.020196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45abfd8e-876c-49f2-965e-fc83d7547be3-tmp\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.020324 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.020241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9pgs\" (UniqueName: \"kubernetes.io/projected/45abfd8e-876c-49f2-965e-fc83d7547be3-kube-api-access-g9pgs\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.020324 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.020270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/45abfd8e-876c-49f2-965e-fc83d7547be3-webhook-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.020427 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.020330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/45abfd8e-876c-49f2-965e-fc83d7547be3-manifests\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.120984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.120948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/45abfd8e-876c-49f2-965e-fc83d7547be3-manifests\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.121174 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.120997    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/45abfd8e-876c-49f2-965e-fc83d7547be3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.121174 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.121029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45abfd8e-876c-49f2-965e-fc83d7547be3-tmp\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.121174 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.121056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9pgs\" (UniqueName: \"kubernetes.io/projected/45abfd8e-876c-49f2-965e-fc83d7547be3-kube-api-access-g9pgs\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.121174 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.121076    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/45abfd8e-876c-49f2-965e-fc83d7547be3-webhook-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.121428 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.121406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/45abfd8e-876c-49f2-965e-fc83d7547be3-manifests\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.123417 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.123386    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45abfd8e-876c-49f2-965e-fc83d7547be3-tmp\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.123613 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.123593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/45abfd8e-876c-49f2-965e-fc83d7547be3-webhook-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.123707 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.123689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/45abfd8e-876c-49f2-965e-fc83d7547be3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.136156 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.136054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9pgs\" (UniqueName: \"kubernetes.io/projected/45abfd8e-876c-49f2-965e-fc83d7547be3-kube-api-access-g9pgs\") pod \"opendatahub-operator-controller-manager-749d6986f8-rl9lz\" (UID: \"45abfd8e-876c-49f2-965e-fc83d7547be3\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.188146 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.188114    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:24.470024 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:24.469989    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"]
Sep 15 22:33:24.473603 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:33:24.473573    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45abfd8e_876c_49f2_965e_fc83d7547be3.slice/crio-8d45f91fcee7c04e9f49854d487e939f3727bb8ee633f5f90a9ac8180e46aadc WatchSource:0}: Error finding container 8d45f91fcee7c04e9f49854d487e939f3727bb8ee633f5f90a9ac8180e46aadc: Status 404 returned error can't find the container with id 8d45f91fcee7c04e9f49854d487e939f3727bb8ee633f5f90a9ac8180e46aadc
Sep 15 22:33:25.175290 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:25.175243    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz" event={"ID":"45abfd8e-876c-49f2-965e-fc83d7547be3","Type":"ContainerStarted","Data":"8d45f91fcee7c04e9f49854d487e939f3727bb8ee633f5f90a9ac8180e46aadc"}
Sep 15 22:33:28.190757 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:28.190711    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz" event={"ID":"45abfd8e-876c-49f2-965e-fc83d7547be3","Type":"ContainerStarted","Data":"b28eb9f7a1975fc3cc866d6b5cdc02858cda7ad3ccf5c3cc9a0fb574575f7dc2"}
Sep 15 22:33:29.196242 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:29.196203    2578 generic.go:356] "Generic (PLEG): container finished" podID="45abfd8e-876c-49f2-965e-fc83d7547be3" containerID="b28eb9f7a1975fc3cc866d6b5cdc02858cda7ad3ccf5c3cc9a0fb574575f7dc2" exitCode=0
Sep 15 22:33:29.196691 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:29.196287    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz" event={"ID":"45abfd8e-876c-49f2-965e-fc83d7547be3","Type":"ContainerDied","Data":"b28eb9f7a1975fc3cc866d6b5cdc02858cda7ad3ccf5c3cc9a0fb574575f7dc2"}
Sep 15 22:33:30.201695 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:30.201656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz" event={"ID":"45abfd8e-876c-49f2-965e-fc83d7547be3","Type":"ContainerStarted","Data":"b44106c7a91d857ca8398e98029a29f923c1dcf4eeb105d08a39f9ac06fb9118"}
Sep 15 22:33:30.202086 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:30.201781    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:30.227622 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:30.227560    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz" podStartSLOduration=2.861501586 podStartE2EDuration="7.227543223s" totalImagesPullingTime="3.310406699s" totalInitContainerRuntime="1.055634938s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:33:30.22504805 +0000 UTC m=+349.978672178" watchObservedRunningTime="2026-09-15 22:33:30.227543223 +0000 UTC m=+349.981167349"
Sep 15 22:33:41.208573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:41.208535    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-rl9lz"
Sep 15 22:33:45.963583 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.963550    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"]
Sep 15 22:33:45.972422 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.972396    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:45.975314 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.975294    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 22:33:45.975423 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.975382    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 22:33:45.976728 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.976705    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 15 22:33:45.976840 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.976742    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8xhvt\""
Sep 15 22:33:45.976840 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.976779    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 15 22:33:45.979047 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.978932    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"]
Sep 15 22:33:45.984279 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.984253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-tls-certs\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:45.984394 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.984316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjjtr\" (UniqueName: \"kubernetes.io/projected/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-kube-api-access-xjjtr\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:45.984436 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:45.984405    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-tmp\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:46.085143 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:46.085080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjjtr\" (UniqueName: \"kubernetes.io/projected/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-kube-api-access-xjjtr\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:46.085143 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:46.085147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-tmp\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:46.085354 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:46.085190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-tls-certs\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:46.085354 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:33:46.085302    2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 15 22:33:46.085425 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:33:46.085370    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-tls-certs podName:c7f49c5f-9f71-4029-9e3e-f3977829ecf8 nodeName:}" failed. No retries permitted until 2026-09-15 22:33:46.585353106 +0000 UTC m=+366.338977212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-tls-certs") pod "kube-auth-proxy-7f98985fb-kt9sm" (UID: "c7f49c5f-9f71-4029-9e3e-f3977829ecf8") : secret "kube-auth-proxy-tls" not found
Sep 15 22:33:46.087359 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:46.087337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-tmp\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:46.093969 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:46.093939    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjjtr\" (UniqueName: \"kubernetes.io/projected/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-kube-api-access-xjjtr\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:46.589650 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:46.589611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-tls-certs\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:46.592206 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:46.592187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7f49c5f-9f71-4029-9e3e-f3977829ecf8-tls-certs\") pod \"kube-auth-proxy-7f98985fb-kt9sm\" (UID: \"c7f49c5f-9f71-4029-9e3e-f3977829ecf8\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:46.882709 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:46.882620    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"
Sep 15 22:33:47.161395 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:47.161371    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm"]
Sep 15 22:33:47.163318 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:33:47.163292    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f49c5f_9f71_4029_9e3e_f3977829ecf8.slice/crio-4aa06bcee943cbb37f07d4b5d95e09f89dfeeab28a7516260aaa919250ee2b8c WatchSource:0}: Error finding container 4aa06bcee943cbb37f07d4b5d95e09f89dfeeab28a7516260aaa919250ee2b8c: Status 404 returned error can't find the container with id 4aa06bcee943cbb37f07d4b5d95e09f89dfeeab28a7516260aaa919250ee2b8c
Sep 15 22:33:47.269456 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:47.269421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm" event={"ID":"c7f49c5f-9f71-4029-9e3e-f3977829ecf8","Type":"ContainerStarted","Data":"4aa06bcee943cbb37f07d4b5d95e09f89dfeeab28a7516260aaa919250ee2b8c"}
Sep 15 22:33:50.290168 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:50.290133    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm" event={"ID":"c7f49c5f-9f71-4029-9e3e-f3977829ecf8","Type":"ContainerStarted","Data":"3a448aff6be5a1664b18d0a07514ba754020aba5d1d516a765385e3fa2ea8878"}
Sep 15 22:33:50.312337 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:50.312283    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7f98985fb-kt9sm" podStartSLOduration=2.659926349 podStartE2EDuration="5.312266756s" totalImagesPullingTime="2.652340407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:33:50.310693796 +0000 UTC m=+370.064317924" watchObservedRunningTime="2026-09-15 22:33:50.312266756 +0000 UTC m=+370.065890883"
Sep 15 22:33:52.968276 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:52.968236    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"]
Sep 15 22:33:52.970884 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:52.970865    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:52.974056 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:52.974035    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qgsk5\""
Sep 15 22:33:52.974541 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:52.974522    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 15 22:33:52.984210 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:52.984185    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"]
Sep 15 22:33:53.048861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.048820    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.048861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.048856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.049064 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.048880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.049064 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.048939    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.049064 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.049001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.049064 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.049046    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.049064 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.049063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.049249 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.049112    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.049249 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.049151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k6kc\" (UniqueName: \"kubernetes.io/projected/82dda82c-dff6-4b63-ba02-e4101d73d9ad-kube-api-access-4k6kc\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.149938 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.149895    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.149938 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.149936    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150177 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.149979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150177 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.149997    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150177 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150177 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k6kc\" (UniqueName: \"kubernetes.io/projected/82dda82c-dff6-4b63-ba02-e4101d73d9ad-kube-api-access-4k6kc\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150337 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150337 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150337 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150443 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150516 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150610 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150595    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150673 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150606    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.150768 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.150746    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.152554 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.152530    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.152704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.152686    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.162230 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.162208    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k6kc\" (UniqueName: \"kubernetes.io/projected/82dda82c-dff6-4b63-ba02-e4101d73d9ad-kube-api-access-4k6kc\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.165567 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.165543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82dda82c-dff6-4b63-ba02-e4101d73d9ad-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8\" (UID: \"82dda82c-dff6-4b63-ba02-e4101d73d9ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.280966 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.280888    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:53.555782 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:53.555683    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"]
Sep 15 22:33:53.558415 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:33:53.558384    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82dda82c_dff6_4b63_ba02_e4101d73d9ad.slice/crio-cb395d98d6f9cdc5d38a336bd3c5d67b498315221edad6176bb5563e554cca4b WatchSource:0}: Error finding container cb395d98d6f9cdc5d38a336bd3c5d67b498315221edad6176bb5563e554cca4b: Status 404 returned error can't find the container with id cb395d98d6f9cdc5d38a336bd3c5d67b498315221edad6176bb5563e554cca4b
Sep 15 22:33:54.062516 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.062478    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xpngn"]
Sep 15 22:33:54.065214 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.065194    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:33:54.068126 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.068089    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 15 22:33:54.068263 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.068104    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2qqd8\""
Sep 15 22:33:54.081833 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.081806    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xpngn"]
Sep 15 22:33:54.159582 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.159552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbfcj\" (UniqueName: \"kubernetes.io/projected/f1934af5-9eec-4282-8ab4-b814bf55accb-kube-api-access-wbfcj\") pod \"odh-model-controller-7998c67dc4-xpngn\" (UID: \"f1934af5-9eec-4282-8ab4-b814bf55accb\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:33:54.159718 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.159614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1934af5-9eec-4282-8ab4-b814bf55accb-cert\") pod \"odh-model-controller-7998c67dc4-xpngn\" (UID: \"f1934af5-9eec-4282-8ab4-b814bf55accb\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:33:54.261017 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.260979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1934af5-9eec-4282-8ab4-b814bf55accb-cert\") pod \"odh-model-controller-7998c67dc4-xpngn\" (UID: \"f1934af5-9eec-4282-8ab4-b814bf55accb\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:33:54.261240 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.261153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbfcj\" (UniqueName: \"kubernetes.io/projected/f1934af5-9eec-4282-8ab4-b814bf55accb-kube-api-access-wbfcj\") pod \"odh-model-controller-7998c67dc4-xpngn\" (UID: \"f1934af5-9eec-4282-8ab4-b814bf55accb\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:33:54.263477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.263452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1934af5-9eec-4282-8ab4-b814bf55accb-cert\") pod \"odh-model-controller-7998c67dc4-xpngn\" (UID: \"f1934af5-9eec-4282-8ab4-b814bf55accb\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:33:54.270727 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.270691    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbfcj\" (UniqueName: \"kubernetes.io/projected/f1934af5-9eec-4282-8ab4-b814bf55accb-kube-api-access-wbfcj\") pod \"odh-model-controller-7998c67dc4-xpngn\" (UID: \"f1934af5-9eec-4282-8ab4-b814bf55accb\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:33:54.306683 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.306647    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8" event={"ID":"82dda82c-dff6-4b63-ba02-e4101d73d9ad","Type":"ContainerStarted","Data":"cb395d98d6f9cdc5d38a336bd3c5d67b498315221edad6176bb5563e554cca4b"}
Sep 15 22:33:54.375249 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.375153    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:33:54.617513 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.617472    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lvld8"]
Sep 15 22:33:54.620410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.620383    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:33:54.623457 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.623430    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 15 22:33:54.634849 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.634778    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lvld8"]
Sep 15 22:33:54.656484 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.656454    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xpngn"]
Sep 15 22:33:54.665846 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.665800    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1961fc4-d87b-40ec-86b6-399d13c0b2ef-tls-certs\") pod \"model-serving-api-54d89bd5fc-lvld8\" (UID: \"f1961fc4-d87b-40ec-86b6-399d13c0b2ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:33:54.665988 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.665878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s5nc\" (UniqueName: \"kubernetes.io/projected/f1961fc4-d87b-40ec-86b6-399d13c0b2ef-kube-api-access-6s5nc\") pod \"model-serving-api-54d89bd5fc-lvld8\" (UID: \"f1961fc4-d87b-40ec-86b6-399d13c0b2ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:33:54.767232 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.766855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1961fc4-d87b-40ec-86b6-399d13c0b2ef-tls-certs\") pod \"model-serving-api-54d89bd5fc-lvld8\" (UID: \"f1961fc4-d87b-40ec-86b6-399d13c0b2ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:33:54.767232 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.766908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5nc\" (UniqueName: \"kubernetes.io/projected/f1961fc4-d87b-40ec-86b6-399d13c0b2ef-kube-api-access-6s5nc\") pod \"model-serving-api-54d89bd5fc-lvld8\" (UID: \"f1961fc4-d87b-40ec-86b6-399d13c0b2ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:33:54.767232 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:33:54.767015    2578 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 15 22:33:54.767232 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:33:54.767111    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1961fc4-d87b-40ec-86b6-399d13c0b2ef-tls-certs podName:f1961fc4-d87b-40ec-86b6-399d13c0b2ef nodeName:}" failed. No retries permitted until 2026-09-15 22:33:55.26706897 +0000 UTC m=+375.020693095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f1961fc4-d87b-40ec-86b6-399d13c0b2ef-tls-certs") pod "model-serving-api-54d89bd5fc-lvld8" (UID: "f1961fc4-d87b-40ec-86b6-399d13c0b2ef") : secret "model-serving-api-tls" not found
Sep 15 22:33:54.776521 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:54.776464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s5nc\" (UniqueName: \"kubernetes.io/projected/f1961fc4-d87b-40ec-86b6-399d13c0b2ef-kube-api-access-6s5nc\") pod \"model-serving-api-54d89bd5fc-lvld8\" (UID: \"f1961fc4-d87b-40ec-86b6-399d13c0b2ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:33:54.825209 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:33:54.825178    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1934af5_9eec_4282_8ab4_b814bf55accb.slice/crio-a91b28a067bbd7a9923ed3d25349879ff40422c7131289e27536614024ca2cbd WatchSource:0}: Error finding container a91b28a067bbd7a9923ed3d25349879ff40422c7131289e27536614024ca2cbd: Status 404 returned error can't find the container with id a91b28a067bbd7a9923ed3d25349879ff40422c7131289e27536614024ca2cbd
Sep 15 22:33:55.270849 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:55.270807    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1961fc4-d87b-40ec-86b6-399d13c0b2ef-tls-certs\") pod \"model-serving-api-54d89bd5fc-lvld8\" (UID: \"f1961fc4-d87b-40ec-86b6-399d13c0b2ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:33:55.274023 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:55.273996    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1961fc4-d87b-40ec-86b6-399d13c0b2ef-tls-certs\") pod \"model-serving-api-54d89bd5fc-lvld8\" (UID: \"f1961fc4-d87b-40ec-86b6-399d13c0b2ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:33:55.313194 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:55.313158    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xpngn" event={"ID":"f1934af5-9eec-4282-8ab4-b814bf55accb","Type":"ContainerStarted","Data":"a91b28a067bbd7a9923ed3d25349879ff40422c7131289e27536614024ca2cbd"}
Sep 15 22:33:55.535890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:55.535805    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:33:56.445435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:56.445378    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:33:56.445945 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:56.445472    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:33:56.445945 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:56.445521    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:33:56.649192 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:56.649160    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lvld8"]
Sep 15 22:33:56.652790 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:33:56.652755    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1961fc4_d87b_40ec_86b6_399d13c0b2ef.slice/crio-296b160573ad2483077811251fc01be254113ba6e3184c33cc62c3b8984e40de WatchSource:0}: Error finding container 296b160573ad2483077811251fc01be254113ba6e3184c33cc62c3b8984e40de: Status 404 returned error can't find the container with id 296b160573ad2483077811251fc01be254113ba6e3184c33cc62c3b8984e40de
Sep 15 22:33:57.324072 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:57.324026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lvld8" event={"ID":"f1961fc4-d87b-40ec-86b6-399d13c0b2ef","Type":"ContainerStarted","Data":"296b160573ad2483077811251fc01be254113ba6e3184c33cc62c3b8984e40de"}
Sep 15 22:33:57.326040 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:57.325996    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8" event={"ID":"82dda82c-dff6-4b63-ba02-e4101d73d9ad","Type":"ContainerStarted","Data":"5307729ecac46e4c8cb01f2c7a577fa15892303fe3dbf2f3adbb521387fff78d"}
Sep 15 22:33:57.349384 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:57.349324    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8" podStartSLOduration=2.464646969 podStartE2EDuration="5.349305502s" totalImagesPullingTime="2.884658533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:33:57.34746147 +0000 UTC m=+377.101085598" watchObservedRunningTime="2026-09-15 22:33:57.349305502 +0000 UTC m=+377.102929629"
Sep 15 22:33:58.281257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:58.281218    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:58.286490 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:58.286459    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:58.331345 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:58.331311    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:33:58.332284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:33:58.332257    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8"
Sep 15 22:34:00.342073 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.342032    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xpngn" event={"ID":"f1934af5-9eec-4282-8ab4-b814bf55accb","Type":"ContainerStarted","Data":"e1d0667f2b28191ddf55e9f9f7018cd12cf3046c515ae837795d79bb16e34b4a"}
Sep 15 22:34:00.342531 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.342127    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:34:00.343623 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.343597    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lvld8" event={"ID":"f1961fc4-d87b-40ec-86b6-399d13c0b2ef","Type":"ContainerStarted","Data":"46dc2bc38e1c43a31d7092a34410a12ce274174266e6eff4648a3e1c6c0cacf5"}
Sep 15 22:34:00.360231 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.360170    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xpngn" podStartSLOduration=1.828292175 podStartE2EDuration="6.360152949s" totalImagesPullingTime="4.531860774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:34:00.35983198 +0000 UTC m=+380.113456109" watchObservedRunningTime="2026-09-15 22:34:00.360152949 +0000 UTC m=+380.113777076"
Sep 15 22:34:00.379355 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.379304    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lvld8" podStartSLOduration=3.677430082 podStartE2EDuration="6.379286825s" totalImagesPullingTime="2.701856743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:34:00.377720226 +0000 UTC m=+380.131344353" watchObservedRunningTime="2026-09-15 22:34:00.379286825 +0000 UTC m=+380.132910952"
Sep 15 22:34:00.571105 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.571067    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"]
Sep 15 22:34:00.574417 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.574397    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.577415 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.577386    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 15 22:34:00.577560 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.577416    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 15 22:34:00.577560 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.577489    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fhmxk\""
Sep 15 22:34:00.577660 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.577620    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 15 22:34:00.591273 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.591249    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"]
Sep 15 22:34:00.721705 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.721667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.721705 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.721704    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.721900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.721727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8943a17-8da5-42be-aeee-4f8da22425d4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.721900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.721798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.721900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.721841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f8943a17-8da5-42be-aeee-4f8da22425d4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.721900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.721868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7rbw\" (UniqueName: \"kubernetes.io/projected/f8943a17-8da5-42be-aeee-4f8da22425d4-kube-api-access-h7rbw\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.722028 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.721933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.823140 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.823077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.823140 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.823144    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f8943a17-8da5-42be-aeee-4f8da22425d4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.823395 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.823176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7rbw\" (UniqueName: \"kubernetes.io/projected/f8943a17-8da5-42be-aeee-4f8da22425d4-kube-api-access-h7rbw\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.823395 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.823205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.823395 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.823227    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.823395 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.823271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.823395 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.823322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8943a17-8da5-42be-aeee-4f8da22425d4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.823863 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.823831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.825647 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.825623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.825740 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.825633    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.825896 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.825877    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8943a17-8da5-42be-aeee-4f8da22425d4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.826033 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.826012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f8943a17-8da5-42be-aeee-4f8da22425d4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.849910 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.849876    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7rbw\" (UniqueName: \"kubernetes.io/projected/f8943a17-8da5-42be-aeee-4f8da22425d4-kube-api-access-h7rbw\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.857202 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.857173    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8943a17-8da5-42be-aeee-4f8da22425d4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fflrv\" (UID: \"f8943a17-8da5-42be-aeee-4f8da22425d4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:00.883050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:00.883018    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:01.158354 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:01.158281    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"]
Sep 15 22:34:01.161883 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:34:01.161852    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8943a17_8da5_42be_aeee_4f8da22425d4.slice/crio-3516ae29044be6e7f91ae4f0c6edb3f1c9c6cac6c28033e4e8c919730cbd78e3 WatchSource:0}: Error finding container 3516ae29044be6e7f91ae4f0c6edb3f1c9c6cac6c28033e4e8c919730cbd78e3: Status 404 returned error can't find the container with id 3516ae29044be6e7f91ae4f0c6edb3f1c9c6cac6c28033e4e8c919730cbd78e3
Sep 15 22:34:01.349117 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:01.349073    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv" event={"ID":"f8943a17-8da5-42be-aeee-4f8da22425d4","Type":"ContainerStarted","Data":"3516ae29044be6e7f91ae4f0c6edb3f1c9c6cac6c28033e4e8c919730cbd78e3"}
Sep 15 22:34:01.349532 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:01.349343    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:34:03.129371 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.129336    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"]
Sep 15 22:34:03.132463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.132443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.137139 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.137117    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 15 22:34:03.139514 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.138976    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-sr5jj\""
Sep 15 22:34:03.139514 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.139112    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 15 22:34:03.139514 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.139116    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 15 22:34:03.160243 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.160215    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"]
Sep 15 22:34:03.243045 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.243001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-cert\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.243223 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.243063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzp9r\" (UniqueName: \"kubernetes.io/projected/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-kube-api-access-kzp9r\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.243223 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.243148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-metrics-cert\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.243223 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.243213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-manager-config\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.344686 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.344646    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-manager-config\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.344686 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.344694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-cert\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.344931 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.344740    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzp9r\" (UniqueName: \"kubernetes.io/projected/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-kube-api-access-kzp9r\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.344931 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.344764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-metrics-cert\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.345364 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.345342    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-manager-config\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.347321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.347298    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-metrics-cert\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.347413 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.347396    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-cert\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.367202 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.367172    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzp9r\" (UniqueName: \"kubernetes.io/projected/0da62b8b-6d6b-48b5-a2e5-458a00b4726d-kube-api-access-kzp9r\") pod \"lws-controller-manager-7bd6c5dcdd-hw5jd\" (UID: \"0da62b8b-6d6b-48b5-a2e5-458a00b4726d\") " pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.442143 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.442080    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:03.723686 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:03.723605    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"]
Sep 15 22:34:03.726410 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:34:03.726381    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da62b8b_6d6b_48b5_a2e5_458a00b4726d.slice/crio-01a7c6f0887a9bf6e14dba20ab7bb8b0f5fed53edc8cc666124129bda6e36cd7 WatchSource:0}: Error finding container 01a7c6f0887a9bf6e14dba20ab7bb8b0f5fed53edc8cc666124129bda6e36cd7: Status 404 returned error can't find the container with id 01a7c6f0887a9bf6e14dba20ab7bb8b0f5fed53edc8cc666124129bda6e36cd7
Sep 15 22:34:04.364219 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:04.364182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd" event={"ID":"0da62b8b-6d6b-48b5-a2e5-458a00b4726d","Type":"ContainerStarted","Data":"01a7c6f0887a9bf6e14dba20ab7bb8b0f5fed53edc8cc666124129bda6e36cd7"}
Sep 15 22:34:05.410063 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:05.410028    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:34:05.410455 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:05.410120    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:34:06.380861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:06.379853    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-fflrv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 22:34:06.380861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:06.379916    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv" podUID="f8943a17-8da5-42be-aeee-4f8da22425d4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 22:34:06.380861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:06.380702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv" event={"ID":"f8943a17-8da5-42be-aeee-4f8da22425d4","Type":"ContainerStarted","Data":"a52b0063de44450c0f7d7def16726f69f573b9d26305aff1eee08c7fafd60af9"}
Sep 15 22:34:06.380861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:06.380791    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:06.416863 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:06.416805    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv" podStartSLOduration=2.170928901 podStartE2EDuration="6.416786989s" totalImagesPullingTime="4.245858088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:34:06.414727127 +0000 UTC m=+386.168351254" watchObservedRunningTime="2026-09-15 22:34:06.416786989 +0000 UTC m=+386.170411116"
Sep 15 22:34:07.381332 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:07.381296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd" event={"ID":"0da62b8b-6d6b-48b5-a2e5-458a00b4726d","Type":"ContainerStarted","Data":"341f4c6935afce6cefd567b930490407d121004bf80a9c65f437f046442e938c"}
Sep 15 22:34:07.381518 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:07.381371    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:34:07.382162 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:07.382142    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fflrv"
Sep 15 22:34:07.400952 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:07.400895    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd" podStartSLOduration=1.542330703 podStartE2EDuration="4.400877815s" totalImagesPullingTime="2.858547112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:34:07.399564431 +0000 UTC m=+387.153188558" watchObservedRunningTime="2026-09-15 22:34:07.400877815 +0000 UTC m=+387.154501941"
Sep 15 22:34:11.351872 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:11.351840    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xpngn"
Sep 15 22:34:12.358241 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:12.358210    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lvld8"
Sep 15 22:34:18.388047 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:34:18.388014    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7bd6c5dcdd-hw5jd"
Sep 15 22:40:22.806526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.806486    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"]
Sep 15 22:40:22.809912 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.809885    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:22.813361 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.813334    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-03f258-openshift-default-dockercfg-bnst5\""
Sep 15 22:40:22.825866 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.825836    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"]
Sep 15 22:40:22.932757 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.932719    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cgg2\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-kube-api-access-7cgg2\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:22.932977 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.932774    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-envoy\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:22.932977 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.932801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-podinfo\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:22.932977 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.932849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-credential-socket\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:22.932977 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.932872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-certs\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:22.932977 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.932928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-data\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:22.932977 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.932956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istiod-ca-cert\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:22.933220 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.933141    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-socket\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:22.933364 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:22.933346    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-token\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.034839 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.034802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-socket\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035035 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.034882    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-token\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035035 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.034922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cgg2\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-kube-api-access-7cgg2\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035035 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.034952    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-envoy\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035035 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.034974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-podinfo\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.035073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-credential-socket\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.035139    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-certs\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.035195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-data\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.035224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istiod-ca-cert\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.035269    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-socket\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.035464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-credential-socket\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.035513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-certs\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035655 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.035628    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-data\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.035904 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.035883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istiod-ca-cert\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.037548 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.037527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-podinfo\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.037737 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.037722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-envoy\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.048359 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.048332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-token\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.049642 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.049623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cgg2\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-kube-api-access-7cgg2\") pod \"e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") " pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.120506 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.120419    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:23.400568 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.400538    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"]
Sep 15 22:40:23.402739 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:40:23.402707    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1a1aa4_d1ce_49c3_a406_025acfe74568.slice/crio-6ebac159987db9ee9fc556905e06ca42790067059922d9e025f5a52cac1d0adb WatchSource:0}: Error finding container 6ebac159987db9ee9fc556905e06ca42790067059922d9e025f5a52cac1d0adb: Status 404 returned error can't find the container with id 6ebac159987db9ee9fc556905e06ca42790067059922d9e025f5a52cac1d0adb
Sep 15 22:40:23.404986 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.404947    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:23.405054 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.405027    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:23.405089 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:23.405058    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:24.030481 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.030444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" event={"ID":"9e1a1aa4-d1ce-49c3-a406-025acfe74568","Type":"ContainerStarted","Data":"380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3"}
Sep 15 22:40:24.030946 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.030534    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" event={"ID":"9e1a1aa4-d1ce-49c3-a406-025acfe74568","Type":"ContainerStarted","Data":"6ebac159987db9ee9fc556905e06ca42790067059922d9e025f5a52cac1d0adb"}
Sep 15 22:40:24.052285 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.052226    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" podStartSLOduration=2.052208934 podStartE2EDuration="2.052208934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:24.050960852 +0000 UTC m=+763.804584983" watchObservedRunningTime="2026-09-15 22:40:24.052208934 +0000 UTC m=+763.805833061"
Sep 15 22:40:24.121625 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.121582    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:24.126831 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.126801    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:24.489728 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.489691    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7663-c698bd"]
Sep 15 22:40:24.493151 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.493130    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7663-c698bd"
Sep 15 22:40:24.495738 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.495716    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4fdg9\""
Sep 15 22:40:24.500912 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.500776    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7663-c698bd"]
Sep 15 22:40:24.649249 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.649209    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkxzk\" (UniqueName: \"kubernetes.io/projected/fece96d5-10ee-462a-816d-95aef888c9be-kube-api-access-nkxzk\") pod \"test-curl-7663-c698bd\" (UID: \"fece96d5-10ee-462a-816d-95aef888c9be\") " pod="opendatahub/test-curl-7663-c698bd"
Sep 15 22:40:24.750386 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.750287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkxzk\" (UniqueName: \"kubernetes.io/projected/fece96d5-10ee-462a-816d-95aef888c9be-kube-api-access-nkxzk\") pod \"test-curl-7663-c698bd\" (UID: \"fece96d5-10ee-462a-816d-95aef888c9be\") " pod="opendatahub/test-curl-7663-c698bd"
Sep 15 22:40:24.759710 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.759673    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkxzk\" (UniqueName: \"kubernetes.io/projected/fece96d5-10ee-462a-816d-95aef888c9be-kube-api-access-nkxzk\") pod \"test-curl-7663-c698bd\" (UID: \"fece96d5-10ee-462a-816d-95aef888c9be\") " pod="opendatahub/test-curl-7663-c698bd"
Sep 15 22:40:24.804016 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:24.803972    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7663-c698bd"
Sep 15 22:40:25.035503 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:25.035419    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:25.036576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:25.036557    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:25.082234 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:25.082200    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7663-c698bd"]
Sep 15 22:40:25.084948 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:40:25.084918    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfece96d5_10ee_462a_816d_95aef888c9be.slice/crio-8a6549b3b4566111f43f957856b1cfed33b368f82bdebc35fcdb6859ac6b9c4d WatchSource:0}: Error finding container 8a6549b3b4566111f43f957856b1cfed33b368f82bdebc35fcdb6859ac6b9c4d: Status 404 returned error can't find the container with id 8a6549b3b4566111f43f957856b1cfed33b368f82bdebc35fcdb6859ac6b9c4d
Sep 15 22:40:25.087078 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:25.087057    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:40:26.041516 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.041480    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7663-c698bd" event={"ID":"fece96d5-10ee-462a-816d-95aef888c9be","Type":"ContainerStarted","Data":"aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f"}
Sep 15 22:40:26.041516 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.041524    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7663-c698bd" event={"ID":"fece96d5-10ee-462a-816d-95aef888c9be","Type":"ContainerStarted","Data":"8a6549b3b4566111f43f957856b1cfed33b368f82bdebc35fcdb6859ac6b9c4d"}
Sep 15 22:40:26.071176 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.071118    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7663-c698bd" podStartSLOduration=1.334796002 podStartE2EDuration="2.071085222s" totalImagesPullingTime="736.28922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:26.069516251 +0000 UTC m=+765.823140377" watchObservedRunningTime="2026-09-15 22:40:26.071085222 +0000 UTC m=+765.824709349"
Sep 15 22:40:26.647287 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.647200    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7663-c698bd"]
Sep 15 22:40:26.650960 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.650931    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7663-c698bd"]
Sep 15 22:40:26.767867 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.767835    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7663-7e8186"]
Sep 15 22:40:26.768226 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.768211    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fece96d5-10ee-462a-816d-95aef888c9be" containerName="test-curl-7663-c698bd"
Sep 15 22:40:26.768281 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.768228    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fece96d5-10ee-462a-816d-95aef888c9be" containerName="test-curl-7663-c698bd"
Sep 15 22:40:26.768317 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.768300    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fece96d5-10ee-462a-816d-95aef888c9be" containerName="test-curl-7663-c698bd"
Sep 15 22:40:26.771394 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.771374    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7663-7e8186"
Sep 15 22:40:26.777345 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.777318    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7663-7e8186"]
Sep 15 22:40:26.870621 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.870577    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkp84\" (UniqueName: \"kubernetes.io/projected/6d5eb477-3dfb-4cdb-9e33-101a07d21954-kube-api-access-zkp84\") pod \"test-curl-7663-7e8186\" (UID: \"6d5eb477-3dfb-4cdb-9e33-101a07d21954\") " pod="opendatahub/test-curl-7663-7e8186"
Sep 15 22:40:26.971987 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.971942    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkp84\" (UniqueName: \"kubernetes.io/projected/6d5eb477-3dfb-4cdb-9e33-101a07d21954-kube-api-access-zkp84\") pod \"test-curl-7663-7e8186\" (UID: \"6d5eb477-3dfb-4cdb-9e33-101a07d21954\") " pod="opendatahub/test-curl-7663-7e8186"
Sep 15 22:40:26.980379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:26.980342    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkp84\" (UniqueName: \"kubernetes.io/projected/6d5eb477-3dfb-4cdb-9e33-101a07d21954-kube-api-access-zkp84\") pod \"test-curl-7663-7e8186\" (UID: \"6d5eb477-3dfb-4cdb-9e33-101a07d21954\") " pod="opendatahub/test-curl-7663-7e8186"
Sep 15 22:40:27.046258 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:40:27.046225    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7663-c698bd" container="test-curl-7663-c698bd"
Sep 15 22:40:27.081755 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:27.081706    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7663-7e8186"
Sep 15 22:40:27.356183 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:27.356143    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7663-7e8186"]
Sep 15 22:40:27.358777 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:40:27.358749    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d5eb477_3dfb_4cdb_9e33_101a07d21954.slice/crio-72bbdf8ebfad20abdf7cceff66be0275f1b55da1ce31e53508df6a2b6808b025 WatchSource:0}: Error finding container 72bbdf8ebfad20abdf7cceff66be0275f1b55da1ce31e53508df6a2b6808b025: Status 404 returned error can't find the container with id 72bbdf8ebfad20abdf7cceff66be0275f1b55da1ce31e53508df6a2b6808b025
Sep 15 22:40:28.051646 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:28.051609    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7663-7e8186" event={"ID":"6d5eb477-3dfb-4cdb-9e33-101a07d21954","Type":"ContainerStarted","Data":"452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89"}
Sep 15 22:40:28.051646 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:28.051656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7663-7e8186" event={"ID":"6d5eb477-3dfb-4cdb-9e33-101a07d21954","Type":"ContainerStarted","Data":"72bbdf8ebfad20abdf7cceff66be0275f1b55da1ce31e53508df6a2b6808b025"}
Sep 15 22:40:28.052207 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:28.051807    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7663-c698bd" podUID="fece96d5-10ee-462a-816d-95aef888c9be" containerName="test-curl-7663-c698bd" containerID="cri-o://aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f" gracePeriod=2
Sep 15 22:40:28.072968 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:28.072895    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7663-7e8186" podStartSLOduration=2.072876731 podStartE2EDuration="2.072876731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:28.071002348 +0000 UTC m=+767.824626475" watchObservedRunningTime="2026-09-15 22:40:28.072876731 +0000 UTC m=+767.826500857"
Sep 15 22:40:28.656068 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:28.656029    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7663-7e8186"]
Sep 15 22:40:28.662530 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:28.662499    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7663-7e8186"]
Sep 15 22:40:29.056866 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:40:29.056833    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7663-7e8186" container="test-curl-7663-7e8186"
Sep 15 22:40:29.335086 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.335012    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"]
Sep 15 22:40:29.335593 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.335557    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d5eb477-3dfb-4cdb-9e33-101a07d21954" containerName="test-curl-7663-7e8186"
Sep 15 22:40:29.335593 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.335585    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d5eb477-3dfb-4cdb-9e33-101a07d21954" containerName="test-curl-7663-7e8186"
Sep 15 22:40:29.335774 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.335694    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d5eb477-3dfb-4cdb-9e33-101a07d21954" containerName="test-curl-7663-7e8186"
Sep 15 22:40:29.338849 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.338827    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.345003 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.344971    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 22:40:29.346294 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.346270    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-fe020d-serving-cert\""
Sep 15 22:40:29.346493 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.346300    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tsrzq\""
Sep 15 22:40:29.346576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.346365    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 22:40:29.346576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.346389    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 22:40:29.351038 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.351012    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"]
Sep 15 22:40:29.441473 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.441438    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"]
Sep 15 22:40:29.444760 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.444731    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.447670 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.447645    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-b08870-openshift-default-dockercfg-jqn2p\""
Sep 15 22:40:29.460582 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.460552    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"]
Sep 15 22:40:29.498911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.498868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56267239-c3a7-4494-b01b-c0c73b9fcb1f-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.499134 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.498984    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-metrics-certs\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.499134 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.499019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4grhs\" (UniqueName: \"kubernetes.io/projected/56267239-c3a7-4494-b01b-c0c73b9fcb1f-kube-api-access-4grhs\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.499134 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.499122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-maas-api-tls\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.600369 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600260    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-maas-api-tls\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.600369 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-token\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.600369 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600349    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-envoy\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.600678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600393    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-certs\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.600678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600418    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rks2\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-kube-api-access-9rks2\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.600678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600448    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56267239-c3a7-4494-b01b-c0c73b9fcb1f-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.600678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600496    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-data\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.600678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-podinfo\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.600678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-credential-socket\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.600678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-metrics-certs\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.600678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-socket\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.601031 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4grhs\" (UniqueName: \"kubernetes.io/projected/56267239-c3a7-4494-b01b-c0c73b9fcb1f-kube-api-access-4grhs\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.601031 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.600706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istiod-ca-cert\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.601198 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.601178    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56267239-c3a7-4494-b01b-c0c73b9fcb1f-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.602838 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.602821    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-maas-api-tls\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.602884 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.602871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-metrics-certs\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.612483 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.612452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4grhs\" (UniqueName: \"kubernetes.io/projected/56267239-c3a7-4494-b01b-c0c73b9fcb1f-kube-api-access-4grhs\") pod \"maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.652066 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.652024    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:29.701611 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.701559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-credential-socket\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.701764 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.701633    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-socket\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.701764 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.701662    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istiod-ca-cert\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.701764 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.701732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-token\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.701937 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.701771    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-envoy\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.701937 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.701876    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-certs\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.701937 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.701913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rks2\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-kube-api-access-9rks2\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.702127 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.701984    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-data\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.702127 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.702013    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-podinfo\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.702127 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.702079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-credential-socket\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.702462 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.702123    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-socket\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.703128 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.702802    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istiod-ca-cert\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.703128 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.702925    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-certs\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.703128 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.703059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-data\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.705650 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.705593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-envoy\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.705815 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.705789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-podinfo\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.713590 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.713424    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rks2\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-kube-api-access-9rks2\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.714025 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.713999    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-token\") pod \"e2e-autores-b08870-openshift-default-5559b9969b-czsd2\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") " pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.756994 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.756957    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:29.929145 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:29.929116    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"]
Sep 15 22:40:29.930888 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:40:29.930863    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56267239_c3a7_4494_b01b_c0c73b9fcb1f.slice/crio-2311794286b374393e50c6b6ff8a299226f2a401f99bb719c3b3b19d2e277332 WatchSource:0}: Error finding container 2311794286b374393e50c6b6ff8a299226f2a401f99bb719c3b3b19d2e277332: Status 404 returned error can't find the container with id 2311794286b374393e50c6b6ff8a299226f2a401f99bb719c3b3b19d2e277332
Sep 15 22:40:30.041746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.041719    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"]
Sep 15 22:40:30.044143 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:40:30.044085    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00bcdafe_87c7_4c02_8a26_21ef6add9c79.slice/crio-d67a474913260a632a1411609b01b3c10b9129d53cd07542d2c3447fd31132c6 WatchSource:0}: Error finding container d67a474913260a632a1411609b01b3c10b9129d53cd07542d2c3447fd31132c6: Status 404 returned error can't find the container with id d67a474913260a632a1411609b01b3c10b9129d53cd07542d2c3447fd31132c6
Sep 15 22:40:30.046523 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.046486    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:30.046629 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.046559    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:30.046629 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.046588    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:30.065020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.064979    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" event={"ID":"00bcdafe-87c7-4c02-8a26-21ef6add9c79","Type":"ContainerStarted","Data":"d67a474913260a632a1411609b01b3c10b9129d53cd07542d2c3447fd31132c6"}
Sep 15 22:40:30.066538 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.066510    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq" event={"ID":"56267239-c3a7-4494-b01b-c0c73b9fcb1f","Type":"ContainerStarted","Data":"2311794286b374393e50c6b6ff8a299226f2a401f99bb719c3b3b19d2e277332"}
Sep 15 22:40:30.066722 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.066694    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7663-7e8186" podUID="6d5eb477-3dfb-4cdb-9e33-101a07d21954" containerName="test-curl-7663-7e8186" containerID="cri-o://452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89" gracePeriod=2
Sep 15 22:40:30.219866 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.219835    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7663-c698bd"
Sep 15 22:40:30.409637 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.409602    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkxzk\" (UniqueName: \"kubernetes.io/projected/fece96d5-10ee-462a-816d-95aef888c9be-kube-api-access-nkxzk\") pod \"fece96d5-10ee-462a-816d-95aef888c9be\" (UID: \"fece96d5-10ee-462a-816d-95aef888c9be\") "
Sep 15 22:40:30.412250 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.412208    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fece96d5-10ee-462a-816d-95aef888c9be-kube-api-access-nkxzk" (OuterVolumeSpecName: "kube-api-access-nkxzk") pod "fece96d5-10ee-462a-816d-95aef888c9be" (UID: "fece96d5-10ee-462a-816d-95aef888c9be"). InnerVolumeSpecName "kube-api-access-nkxzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:40:30.510910 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.510807    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkxzk\" (UniqueName: \"kubernetes.io/projected/fece96d5-10ee-462a-816d-95aef888c9be-kube-api-access-nkxzk\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:40:30.766414 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:30.766328    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fece96d5-10ee-462a-816d-95aef888c9be" path="/var/lib/kubelet/pods/fece96d5-10ee-462a-816d-95aef888c9be/volumes"
Sep 15 22:40:31.075432 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.075345    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" event={"ID":"00bcdafe-87c7-4c02-8a26-21ef6add9c79","Type":"ContainerStarted","Data":"3e4230da6b07dfb84ecc6c3e6af24e5c55e60ad44c502af12fa805849eb69be4"}
Sep 15 22:40:31.077200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.077166    2578 generic.go:356] "Generic (PLEG): container finished" podID="fece96d5-10ee-462a-816d-95aef888c9be" containerID="aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f" exitCode=137
Sep 15 22:40:31.077353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.077213    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7663-c698bd"
Sep 15 22:40:31.077353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.077235    2578 scope.go:117] "RemoveContainer" containerID="aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f"
Sep 15 22:40:31.087085 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.087047    2578 scope.go:117] "RemoveContainer" containerID="aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f"
Sep 15 22:40:31.088048 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:40:31.088014    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f\": container with ID starting with aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f not found: ID does not exist" containerID="aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f"
Sep 15 22:40:31.088188 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.088059    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f"} err="failed to get container status \"aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f\": rpc error: code = NotFound desc = could not find container \"aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f\": container with ID starting with aa6719610019369be048b14df2a88697373e38e86b71e6e28639b25d18bb190f not found: ID does not exist"
Sep 15 22:40:31.098323 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.098270    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" podStartSLOduration=2.098252163 podStartE2EDuration="2.098252163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:31.096417985 +0000 UTC m=+770.850042111" watchObservedRunningTime="2026-09-15 22:40:31.098252163 +0000 UTC m=+770.851876289"
Sep 15 22:40:31.758061 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.758020    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:31.764289 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.764260    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:31.803028 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.802991    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"]
Sep 15 22:40:31.815532 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.815500    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.819125 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.819071    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"]
Sep 15 22:40:31.819276 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.819070    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-33ff1a95-openshift-default-dockercfg-7s2f6\""
Sep 15 22:40:31.823142 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.822988    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-certs\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.823142 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.823030    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-envoy\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.823142 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.823055    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-podinfo\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.823419 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.823200    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-socket\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.823419 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.823241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-credential-socket\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.823419 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.823279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-data\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.823419 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.823343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-token\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.823419 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.823364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h9m8\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-kube-api-access-9h9m8\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.823600 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.823447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istiod-ca-cert\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.924305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-token\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.924352    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9m8\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-kube-api-access-9h9m8\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.924398    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istiod-ca-cert\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.925155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-certs\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.925190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-envoy\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.925217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-podinfo\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.925272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-socket\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.925296    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-credential-socket\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.925321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-data\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.925492    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istiod-ca-cert\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.925801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-certs\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.925990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.926001    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-data\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.927901 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.926277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-socket\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.927901 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.926553    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-credential-socket\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.929859 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.929796    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-podinfo\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.930693 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.930642    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-envoy\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.937181 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.937148    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9m8\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-kube-api-access-9h9m8\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:31.937344 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:31.937217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-token\") pod \"e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") " pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:32.084462 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.084426    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:32.085534 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.085511    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:40:32.128770 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.128561    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:32.423331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.423297    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7663-7e8186"
Sep 15 22:40:32.428992 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.428962    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkp84\" (UniqueName: \"kubernetes.io/projected/6d5eb477-3dfb-4cdb-9e33-101a07d21954-kube-api-access-zkp84\") pod \"6d5eb477-3dfb-4cdb-9e33-101a07d21954\" (UID: \"6d5eb477-3dfb-4cdb-9e33-101a07d21954\") "
Sep 15 22:40:32.430894 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.430866    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d5eb477-3dfb-4cdb-9e33-101a07d21954-kube-api-access-zkp84" (OuterVolumeSpecName: "kube-api-access-zkp84") pod "6d5eb477-3dfb-4cdb-9e33-101a07d21954" (UID: "6d5eb477-3dfb-4cdb-9e33-101a07d21954"). InnerVolumeSpecName "kube-api-access-zkp84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:40:32.531336 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.531279    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkp84\" (UniqueName: \"kubernetes.io/projected/6d5eb477-3dfb-4cdb-9e33-101a07d21954-kube-api-access-zkp84\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:40:32.566163 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.566128    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"]
Sep 15 22:40:32.569142 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:40:32.569108    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f1c8e3_d9ae_4854_9872_4dab7978bcd7.slice/crio-c1a7fb6d3ab592eaafa850032e878f568fa969715c55f19a625233d101db5d99 WatchSource:0}: Error finding container c1a7fb6d3ab592eaafa850032e878f568fa969715c55f19a625233d101db5d99: Status 404 returned error can't find the container with id c1a7fb6d3ab592eaafa850032e878f568fa969715c55f19a625233d101db5d99
Sep 15 22:40:32.571431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.571398    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:32.571497 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.571467    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:32.571582 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.571496    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:32.763576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:32.763540    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d5eb477-3dfb-4cdb-9e33-101a07d21954" path="/var/lib/kubelet/pods/6d5eb477-3dfb-4cdb-9e33-101a07d21954/volumes"
Sep 15 22:40:33.091004 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.090968    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d5eb477-3dfb-4cdb-9e33-101a07d21954" containerID="452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89" exitCode=137
Sep 15 22:40:33.091004 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.091014    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7663-7e8186"
Sep 15 22:40:33.091549 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.091076    2578 scope.go:117] "RemoveContainer" containerID="452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89"
Sep 15 22:40:33.094022 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.093347    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" event={"ID":"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7","Type":"ContainerStarted","Data":"aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a"}
Sep 15 22:40:33.094022 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.093408    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" event={"ID":"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7","Type":"ContainerStarted","Data":"c1a7fb6d3ab592eaafa850032e878f568fa969715c55f19a625233d101db5d99"}
Sep 15 22:40:33.095815 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.095782    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq" event={"ID":"56267239-c3a7-4494-b01b-c0c73b9fcb1f","Type":"ContainerStarted","Data":"9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247"}
Sep 15 22:40:33.095954 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.095929    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:33.100677 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.100658    2578 scope.go:117] "RemoveContainer" containerID="452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89"
Sep 15 22:40:33.100992 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:40:33.100972    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89\": container with ID starting with 452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89 not found: ID does not exist" containerID="452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89"
Sep 15 22:40:33.101058 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.100997    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89"} err="failed to get container status \"452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89\": rpc error: code = NotFound desc = could not find container \"452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89\": container with ID starting with 452211c1f9be9d2ea167b25e55c23cad26edfc64a9a18d79a9e0dd842b395b89 not found: ID does not exist"
Sep 15 22:40:33.119891 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.119832    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" podStartSLOduration=2.119816002 podStartE2EDuration="2.119816002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:33.119250625 +0000 UTC m=+772.872874753" watchObservedRunningTime="2026-09-15 22:40:33.119816002 +0000 UTC m=+772.873440129"
Sep 15 22:40:33.129505 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.129468    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:33.134910 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.134884    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:33.140770 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:33.140704    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq" podStartSLOduration=1.730050403 podStartE2EDuration="4.140687278s" totalImagesPullingTime="2.410636875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:33.139585699 +0000 UTC m=+772.893209827" watchObservedRunningTime="2026-09-15 22:40:33.140687278 +0000 UTC m=+772.894311405"
Sep 15 22:40:34.101441 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:34.101406    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:34.102368 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:34.102350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:39.116529 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:39.116487    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:40:44.158707 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:44.158613    2578 generic.go:356] "Generic (PLEG): container finished" podID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerID="380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3" exitCode=0
Sep 15 22:40:44.158707 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:44.158653    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" event={"ID":"9e1a1aa4-d1ce-49c3-a406-025acfe74568","Type":"ContainerDied","Data":"380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3"}
Sep 15 22:40:44.159204 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:44.159032    2578 scope.go:117] "RemoveContainer" containerID="380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3"
Sep 15 22:40:44.160173 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:44.160141    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:44.160251 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:44.160208    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:44.160251 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:44.160240    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:45.166226 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:45.166189    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" event={"ID":"9e1a1aa4-d1ce-49c3-a406-025acfe74568","Type":"ContainerStarted","Data":"d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be"}
Sep 15 22:40:45.938010 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:45.937960    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"]
Sep 15 22:40:45.942702 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:45.942665    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:45.946723 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:45.946496    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-765e6d-openshift-default-dockercfg-w562g\""
Sep 15 22:40:45.958055 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:45.958017    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"]
Sep 15 22:40:46.062322 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.062279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-credential-socket\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.062322 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.062320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbnvx\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-kube-api-access-bbnvx\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.062586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.062353    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-certs\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.062586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.062373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-envoy\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.062586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.062460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-data\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.062586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.062532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89221884-c915-4606-a1ac-3d684675fa1b-istio-podinfo\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.062586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.062562    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89221884-c915-4606-a1ac-3d684675fa1b-istiod-ca-cert\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.062911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.062621    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-socket\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.062911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.062644    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-istio-token\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.121806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.121324    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:46.121806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.121466    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 15 22:40:46.121806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.121514    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 15 22:40:46.163770 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.163718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89221884-c915-4606-a1ac-3d684675fa1b-istio-podinfo\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.164226 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89221884-c915-4606-a1ac-3d684675fa1b-istiod-ca-cert\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.164306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164290    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-socket\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.164340 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164323    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-istio-token\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.164426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-credential-socket\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.164474 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbnvx\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-kube-api-access-bbnvx\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.164531 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-certs\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.164531 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-envoy\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.164630 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-data\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.164758 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164731    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-socket\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.165105 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164824    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89221884-c915-4606-a1ac-3d684675fa1b-istiod-ca-cert\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.165105 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.164892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-data\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.165255 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.165089    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-certs\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.165255 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.165125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-credential-socket\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.166409 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.166391    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89221884-c915-4606-a1ac-3d684675fa1b-istio-podinfo\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.167037 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.166993    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-envoy\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.172635 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.172583    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-istio-token\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.172745 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.172734    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbnvx\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-kube-api-access-bbnvx\") pod \"e2e-inf-b-765e6d-openshift-default-874566f4b-699h4\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") " pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.257392 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.257301    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:46.591350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.591199    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"]
Sep 15 22:40:46.607453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.607403    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:46.607588 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.607561    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:46.607650 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:46.607615    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:47.121802 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:47.121332    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 15 22:40:47.121802 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:47.121398    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 15 22:40:47.177696 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:47.177657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" event={"ID":"89221884-c915-4606-a1ac-3d684675fa1b","Type":"ContainerStarted","Data":"9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa"}
Sep 15 22:40:47.178177 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:47.177717    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" event={"ID":"89221884-c915-4606-a1ac-3d684675fa1b","Type":"ContainerStarted","Data":"df730a30728a9b4ead87fed0cb2447ae26af0e5a1e5c1224d9f0752def195583"}
Sep 15 22:40:47.200863 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:47.200793    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" podStartSLOduration=2.200770791 podStartE2EDuration="2.200770791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:47.198069465 +0000 UTC m=+786.951693592" watchObservedRunningTime="2026-09-15 22:40:47.200770791 +0000 UTC m=+786.954394910"
Sep 15 22:40:47.258201 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:47.258166    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:47.259292 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:47.259264    2578 patch_prober.go:28] interesting pod/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 15 22:40:47.259434 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:47.259315    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 15 22:40:48.122625 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:48.122582    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 15 22:40:48.122852 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:48.122668    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 15 22:40:48.262635 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:48.262598    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:49.121585 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:49.121542    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 15 22:40:49.121765 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:49.121612    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 15 22:40:49.188874 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:49.188838    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:49.189917 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:49.189891    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:40:50.121203 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:50.121153    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 15 22:40:50.121683 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:50.121228    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 15 22:40:50.143323 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:50.143279    2578 patch_prober.go:28] interesting pod/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 15 22:40:50.143520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:50.143349    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 15 22:40:51.126368 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:51.126336    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:51.126841 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:51.126759    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:51.127617 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:51.127596    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:40:52.851888 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:52.851853    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"]
Sep 15 22:40:52.854437 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:52.854416    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:52.857161 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:52.857139    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-765e6d-serving-cert\""
Sep 15 22:40:52.866348 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:52.866303    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"]
Sep 15 22:40:52.936675 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:52.936637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d4f5b2-d34b-4dbd-b23e-547072b855d7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:52.936675 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:52.936679    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:52.936902 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:52.936796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d45w\" (UniqueName: \"kubernetes.io/projected/04d4f5b2-d34b-4dbd-b23e-547072b855d7-kube-api-access-9d45w\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:52.936992 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:52.936890    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-metrics-certs\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:53.038166 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.038123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d45w\" (UniqueName: \"kubernetes.io/projected/04d4f5b2-d34b-4dbd-b23e-547072b855d7-kube-api-access-9d45w\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:53.038345 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.038205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-metrics-certs\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:53.038345 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.038245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d4f5b2-d34b-4dbd-b23e-547072b855d7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:53.038345 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.038267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:53.038451 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:40:53.038389    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-serving-cert: secret "maas-api-e2e-inf-b-765e6d-serving-cert" not found
Sep 15 22:40:53.038485 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:40:53.038457    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls podName:04d4f5b2-d34b-4dbd-b23e-547072b855d7 nodeName:}" failed. No retries permitted until 2026-09-15 22:40:53.538437736 +0000 UTC m=+793.292061841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls") pod "maas-api-e2e-inf-b-765e6d-568854d466-bwmvf" (UID: "04d4f5b2-d34b-4dbd-b23e-547072b855d7") : secret "maas-api-e2e-inf-b-765e6d-serving-cert" not found
Sep 15 22:40:53.038900 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.038880    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d4f5b2-d34b-4dbd-b23e-547072b855d7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:53.040836 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.040817    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-metrics-certs\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:53.047686 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.047658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d45w\" (UniqueName: \"kubernetes.io/projected/04d4f5b2-d34b-4dbd-b23e-547072b855d7-kube-api-access-9d45w\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:53.218537 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.218495    2578 generic.go:356] "Generic (PLEG): container finished" podID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerID="aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a" exitCode=0
Sep 15 22:40:53.218725 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.218575    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" event={"ID":"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7","Type":"ContainerDied","Data":"aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a"}
Sep 15 22:40:53.219203 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.219179    2578 scope.go:117] "RemoveContainer" containerID="aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a"
Sep 15 22:40:53.220901 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.220832    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:53.221018 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.220934    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:53.221018 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.220995    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:40:53.544128 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:53.544023    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:53.544384 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:40:53.544256    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-serving-cert: secret "maas-api-e2e-inf-b-765e6d-serving-cert" not found
Sep 15 22:40:53.544384 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:40:53.544340    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls podName:04d4f5b2-d34b-4dbd-b23e-547072b855d7 nodeName:}" failed. No retries permitted until 2026-09-15 22:40:54.544319418 +0000 UTC m=+794.297943547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls") pod "maas-api-e2e-inf-b-765e6d-568854d466-bwmvf" (UID: "04d4f5b2-d34b-4dbd-b23e-547072b855d7") : secret "maas-api-e2e-inf-b-765e6d-serving-cert" not found
Sep 15 22:40:54.228826 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:54.228774    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" event={"ID":"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7","Type":"ContainerStarted","Data":"9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891"}
Sep 15 22:40:54.555455 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:54.555350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:54.557872 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:54.557852    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-765e6d-568854d466-bwmvf\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:54.668047 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:54.668005    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:54.987914 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:54.987879    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"]
Sep 15 22:40:54.991002 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:40:54.990970    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d4f5b2_d34b_4dbd_b23e_547072b855d7.slice/crio-7e358042d87a9e37815d80ba23876b2bcd6976cbcecc1184b044858c7251930a WatchSource:0}: Error finding container 7e358042d87a9e37815d80ba23876b2bcd6976cbcecc1184b044858c7251930a: Status 404 returned error can't find the container with id 7e358042d87a9e37815d80ba23876b2bcd6976cbcecc1184b044858c7251930a
Sep 15 22:40:55.130201 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:55.129398    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:40:55.130201 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:55.129827    2578 patch_prober.go:28] interesting pod/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 15 22:40:55.130201 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:55.129881    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 15 22:40:55.238439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:55.238327    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf" event={"ID":"04d4f5b2-d34b-4dbd-b23e-547072b855d7","Type":"ContainerStarted","Data":"7e358042d87a9e37815d80ba23876b2bcd6976cbcecc1184b044858c7251930a"}
Sep 15 22:40:56.129315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:56.129269    2578 patch_prober.go:28] interesting pod/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 15 22:40:56.129538 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:56.129342    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 15 22:40:56.246631 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:56.246589    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf" event={"ID":"04d4f5b2-d34b-4dbd-b23e-547072b855d7","Type":"ContainerStarted","Data":"6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1"}
Sep 15 22:40:56.247035 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:56.246673    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:40:56.268939 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:56.268875    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf" podStartSLOduration=3.909265868 podStartE2EDuration="4.268858771s" totalImagesPullingTime="359.592903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:56.26748858 +0000 UTC m=+796.021112703" watchObservedRunningTime="2026-09-15 22:40:56.268858771 +0000 UTC m=+796.022482898"
Sep 15 22:40:57.129117 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:57.129045    2578 patch_prober.go:28] interesting pod/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 15 22:40:57.129463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:57.129428    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 15 22:40:58.129783 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:58.129738    2578 patch_prober.go:28] interesting pod/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 15 22:40:58.130184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:58.129831    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 15 22:40:59.129755 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:59.129708    2578 patch_prober.go:28] interesting pod/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 15 22:40:59.129939 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:40:59.129793    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 15 22:41:00.134372 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:00.134336    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:41:00.134960 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:00.134928    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:41:00.135931 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:00.135907    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:41:01.274719 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:01.274685    2578 generic.go:356] "Generic (PLEG): container finished" podID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerID="3e4230da6b07dfb84ecc6c3e6af24e5c55e60ad44c502af12fa805849eb69be4" exitCode=0
Sep 15 22:41:01.275169 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:01.274765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" event={"ID":"00bcdafe-87c7-4c02-8a26-21ef6add9c79","Type":"ContainerDied","Data":"3e4230da6b07dfb84ecc6c3e6af24e5c55e60ad44c502af12fa805849eb69be4"}
Sep 15 22:41:01.275315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:01.275298    2578 scope.go:117] "RemoveContainer" containerID="3e4230da6b07dfb84ecc6c3e6af24e5c55e60ad44c502af12fa805849eb69be4"
Sep 15 22:41:01.276426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:01.276396    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:01.276533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:01.276478    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:01.276533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:01.276527    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:01.377552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:01.377504    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"]
Sep 15 22:41:02.084449 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.084404    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:41:02.260023 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.259993    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:41:02.282228 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.282183    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" event={"ID":"00bcdafe-87c7-4c02-8a26-21ef6add9c79","Type":"ContainerStarted","Data":"2e2200f58083f837331078abf3bb358a8f97ec77756bb68e40cc611b991f6072"}
Sep 15 22:41:02.282667 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.282282    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy" containerID="cri-o://9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891" gracePeriod=30
Sep 15 22:41:02.327504 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.327469    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"]
Sep 15 22:41:02.331143 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.331004    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.334996 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.334385    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-36d266fd-gw-openshift-default-dockercfg-8gmsj\""
Sep 15 22:41:02.335848 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.335818    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"]
Sep 15 22:41:02.446426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.446331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-podinfo\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.446426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.446374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-credential-socket\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.446426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.446401    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-token\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.446730 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.446507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623da280-afc2-49c1-848e-d1ecc9d232a4-istiod-ca-cert\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.446730 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.446545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-data\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.446730 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.446578    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-certs\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.446730 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.446601    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tmsc\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-kube-api-access-8tmsc\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.446730 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.446628    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-socket\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.446730 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.446652    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-envoy\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.547959 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.547918    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-data\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548218 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.547976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-certs\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548218 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548010    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tmsc\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-kube-api-access-8tmsc\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548218 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548042    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-socket\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548218 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-envoy\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548218 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-podinfo\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548218 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548180    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-credential-socket\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-token\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548313    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623da280-afc2-49c1-848e-d1ecc9d232a4-istiod-ca-cert\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548439    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-data\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-certs\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548741 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-socket\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.548798 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.548782    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-credential-socket\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.549088 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.549065    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623da280-afc2-49c1-848e-d1ecc9d232a4-istiod-ca-cert\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.550628 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.550605    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-envoy\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.550829 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.550813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-podinfo\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.558080 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.558044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-token\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.558280 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.558258    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tmsc\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-kube-api-access-8tmsc\") pod \"e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") " pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.648129 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.647984    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:02.757418 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.757383    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:41:02.758264 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.758238    2578 patch_prober.go:28] interesting pod/e2e-autores-b08870-openshift-default-5559b9969b-czsd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 15 22:41:02.758399 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.758287    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 15 22:41:02.965386 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.965351    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"]
Sep 15 22:41:02.967142 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:41:02.967112    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623da280_afc2_49c1_848e_d1ecc9d232a4.slice/crio-6ff137a1482548f4a346f514ef05e9935bad2c6adbff23fc6099f8c21a653749 WatchSource:0}: Error finding container 6ff137a1482548f4a346f514ef05e9935bad2c6adbff23fc6099f8c21a653749: Status 404 returned error can't find the container with id 6ff137a1482548f4a346f514ef05e9935bad2c6adbff23fc6099f8c21a653749
Sep 15 22:41:02.969567 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.969530    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:02.969658 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.969619    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:02.969658 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:02.969648    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:03.289308 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:03.289202    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" event={"ID":"623da280-afc2-49c1-848e-d1ecc9d232a4","Type":"ContainerStarted","Data":"138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3"}
Sep 15 22:41:03.289704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:03.289336    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" event={"ID":"623da280-afc2-49c1-848e-d1ecc9d232a4","Type":"ContainerStarted","Data":"6ff137a1482548f4a346f514ef05e9935bad2c6adbff23fc6099f8c21a653749"}
Sep 15 22:41:03.317381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:03.317310    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" podStartSLOduration=1.317284136 podStartE2EDuration="1.317284136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:41:03.314241431 +0000 UTC m=+803.067865561" watchObservedRunningTime="2026-09-15 22:41:03.317284136 +0000 UTC m=+803.070908264"
Sep 15 22:41:03.648544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:03.648448    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:03.758557 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:03.758515    2578 patch_prober.go:28] interesting pod/e2e-autores-b08870-openshift-default-5559b9969b-czsd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 15 22:41:03.758731 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:03.758585    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 15 22:41:04.654279 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:04.654247    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:04.758042 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:04.758006    2578 patch_prober.go:28] interesting pod/e2e-autores-b08870-openshift-default-5559b9969b-czsd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 15 22:41:04.758352 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:04.758318    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 15 22:41:05.304917 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:05.304838    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:05.314770 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:05.314732    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:05.757686 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:05.757652    2578 patch_prober.go:28] interesting pod/e2e-autores-b08870-openshift-default-5559b9969b-czsd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 15 22:41:05.758417 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:05.757715    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 15 22:41:06.757526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:06.757491    2578 patch_prober.go:28] interesting pod/e2e-autores-b08870-openshift-default-5559b9969b-czsd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 15 22:41:06.757710 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:06.757546    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 15 22:41:07.661470 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.661430    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:41:07.757475 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.757438    2578 patch_prober.go:28] interesting pod/e2e-autores-b08870-openshift-default-5559b9969b-czsd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 15 22:41:07.757645 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.757499    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 15 22:41:07.830356 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830319    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-data\") pod \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") "
Sep 15 22:41:07.830553 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830426    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-certs\") pod \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") "
Sep 15 22:41:07.830553 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830479    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istiod-ca-cert\") pod \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") "
Sep 15 22:41:07.830553 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830505    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-credential-socket\") pod \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") "
Sep 15 22:41:07.830553 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830542    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-token\") pod \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") "
Sep 15 22:41:07.830767 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830580    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-podinfo\") pod \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") "
Sep 15 22:41:07.830767 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830610    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-socket\") pod \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") "
Sep 15 22:41:07.830767 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830626    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" (UID: "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:07.830767 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830702    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" (UID: "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:07.830961 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830783    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h9m8\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-kube-api-access-9h9m8\") pod \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") "
Sep 15 22:41:07.830961 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830842    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" (UID: "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:41:07.830961 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830850    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-envoy\") pod \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\" (UID: \"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7\") "
Sep 15 22:41:07.830961 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830914    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" (UID: "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:07.830961 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.830927    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" (UID: "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:07.831342 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.831299    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:07.831473 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.831437    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:07.831473 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.831456    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:07.831608 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.831470    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:07.831608 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.831487    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:07.833176 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.833149    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" (UID: "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:41:07.833288 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.833185    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" (UID: "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:41:07.833426 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.833400    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" (UID: "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:07.833511 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.833433    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-kube-api-access-9h9m8" (OuterVolumeSpecName: "kube-api-access-9h9m8") pod "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" (UID: "c1f1c8e3-d9ae-4854-9872-4dab7978bcd7"). InnerVolumeSpecName "kube-api-access-9h9m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:41:07.932807 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.932600    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:07.932807 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.932647    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:07.932807 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.932659    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:07.932807 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:07.932671    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h9m8\" (UniqueName: \"kubernetes.io/projected/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7-kube-api-access-9h9m8\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:08.326158 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.326123    2578 generic.go:356] "Generic (PLEG): container finished" podID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerID="9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891" exitCode=0
Sep 15 22:41:08.326158 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.326167    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"
Sep 15 22:41:08.326390 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.326202    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" event={"ID":"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7","Type":"ContainerDied","Data":"9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891"}
Sep 15 22:41:08.326390 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.326272    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz" event={"ID":"c1f1c8e3-d9ae-4854-9872-4dab7978bcd7","Type":"ContainerDied","Data":"c1a7fb6d3ab592eaafa850032e878f568fa969715c55f19a625233d101db5d99"}
Sep 15 22:41:08.326390 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.326305    2578 scope.go:117] "RemoveContainer" containerID="9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891"
Sep 15 22:41:08.336748 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.336728    2578 scope.go:117] "RemoveContainer" containerID="aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a"
Sep 15 22:41:08.347272 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.347246    2578 scope.go:117] "RemoveContainer" containerID="9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891"
Sep 15 22:41:08.347650 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:41:08.347628    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891\": container with ID starting with 9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891 not found: ID does not exist" containerID="9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891"
Sep 15 22:41:08.347718 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.347660    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891"} err="failed to get container status \"9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891\": rpc error: code = NotFound desc = could not find container \"9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891\": container with ID starting with 9c2cec67e34e47877177bb8d5e311e44b6165bc8dd1536f98bfdd087dd9f7891 not found: ID does not exist"
Sep 15 22:41:08.347718 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.347681    2578 scope.go:117] "RemoveContainer" containerID="aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a"
Sep 15 22:41:08.348043 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:41:08.348024    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a\": container with ID starting with aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a not found: ID does not exist" containerID="aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a"
Sep 15 22:41:08.348084 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.348051    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a"} err="failed to get container status \"aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a\": rpc error: code = NotFound desc = could not find container \"aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a\": container with ID starting with aecda634f53464415774838450c241ab60361fcf41c729861d3370860aa8d66a not found: ID does not exist"
Sep 15 22:41:08.357988 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.357955    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"]
Sep 15 22:41:08.377347 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.377313    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-33ff1a95-openshift-default-5c69b8cd5b-lnncz"]
Sep 15 22:41:08.763457 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.763425    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" path="/var/lib/kubelet/pods/c1f1c8e3-d9ae-4854-9872-4dab7978bcd7/volumes"
Sep 15 22:41:08.763933 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.763894    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:41:08.764312 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.764293    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:41:08.765077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:08.765060    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:41:13.352123 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:13.352075    2578 generic.go:356] "Generic (PLEG): container finished" podID="89221884-c915-4606-a1ac-3d684675fa1b" containerID="9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa" exitCode=0
Sep 15 22:41:13.352668 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:13.352146    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" event={"ID":"89221884-c915-4606-a1ac-3d684675fa1b","Type":"ContainerDied","Data":"9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa"}
Sep 15 22:41:13.352734 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:13.352715    2578 scope.go:117] "RemoveContainer" containerID="9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa"
Sep 15 22:41:13.354008 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:13.353976    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:13.354118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:13.354051    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:13.354118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:13.354083    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:14.360266 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:14.360234    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" event={"ID":"89221884-c915-4606-a1ac-3d684675fa1b","Type":"ContainerStarted","Data":"d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324"}
Sep 15 22:41:15.258382 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:15.258350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:41:15.259078 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:15.259056    2578 patch_prober.go:28] interesting pod/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 15 22:41:15.259190 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:15.259123    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 15 22:41:16.258406 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:16.258364    2578 patch_prober.go:28] interesting pod/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 15 22:41:16.258797 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:16.258432    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 15 22:41:17.258458 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:17.258418    2578 patch_prober.go:28] interesting pod/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 15 22:41:17.258860 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:17.258488    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 15 22:41:18.258109 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:18.258061    2578 patch_prober.go:28] interesting pod/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 15 22:41:18.258304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:18.258151    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 15 22:41:19.189423 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:19.189376    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:41:19.258296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:19.258258    2578 patch_prober.go:28] interesting pod/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 15 22:41:19.258479 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:19.258316    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 15 22:41:20.263114 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:20.263067    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:41:20.264039 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:20.264022    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:41:24.411459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:24.411423    2578 generic.go:356] "Generic (PLEG): container finished" podID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerID="138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3" exitCode=0
Sep 15 22:41:24.411917 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:24.411476    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" event={"ID":"623da280-afc2-49c1-848e-d1ecc9d232a4","Type":"ContainerDied","Data":"138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3"}
Sep 15 22:41:24.411917 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:24.411903    2578 scope.go:117] "RemoveContainer" containerID="138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3"
Sep 15 22:41:24.413050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:24.413022    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:24.413119 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:24.413106    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:24.413160 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:24.413141    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:25.421931 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:25.421887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" event={"ID":"623da280-afc2-49c1-848e-d1ecc9d232a4","Type":"ContainerStarted","Data":"ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec"}
Sep 15 22:41:25.648588 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:25.648547    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:25.648846 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:25.648821    2578 patch_prober.go:28] interesting pod/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 15 22:41:25.648903 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:25.648885    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 15 22:41:26.648474 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:26.648427    2578 patch_prober.go:28] interesting pod/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 15 22:41:26.648907 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:26.648496    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 15 22:41:27.649164 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:27.649123    2578 patch_prober.go:28] interesting pod/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 15 22:41:27.649558 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:27.649181    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 15 22:41:28.648904 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:28.648868    2578 patch_prober.go:28] interesting pod/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 15 22:41:28.649087 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:28.648929    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 15 22:41:29.648854 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:29.648810    2578 patch_prober.go:28] interesting pod/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 15 22:41:29.649292 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:29.648887    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 15 22:41:30.652916 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:30.652877    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:30.653367 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:30.653129    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:30.654069 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:30.654053    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:34.253170 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.253136    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"]
Sep 15 22:41:34.253561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.253518    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy"
Sep 15 22:41:34.253561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.253530    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy"
Sep 15 22:41:34.253636 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.253602    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy"
Sep 15 22:41:34.253636 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.253611    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy"
Sep 15 22:41:34.255698 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.255676    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.258775 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.258752    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-03f258\"/\"kube-root-ca.crt\""
Sep 15 22:41:34.260303 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.260284    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-03f258\"/\"simulator-w4-03f258-kserve-self-signed-certs\""
Sep 15 22:41:34.260431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.260283    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-03f258\"/\"default-dockercfg-km8rr\""
Sep 15 22:41:34.260431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.260341    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-03f258\"/\"openshift-service-ca.crt\""
Sep 15 22:41:34.266746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.266720    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"]
Sep 15 22:41:34.414102 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.414062    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tls-certs\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.414295 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.414181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-model-cache\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.414295 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.414252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tmp-dir\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.414295 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.414276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wjzs\" (UniqueName: \"kubernetes.io/projected/8b9ef65a-1d7f-438b-9503-cfb8b0963503-kube-api-access-5wjzs\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.414397 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.414295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-home\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.414397 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.414370    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-dshm\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.514993 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.514896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tls-certs\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.514993 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.514948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-model-cache\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.514993 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.514992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tmp-dir\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.515302 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.515015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wjzs\" (UniqueName: \"kubernetes.io/projected/8b9ef65a-1d7f-438b-9503-cfb8b0963503-kube-api-access-5wjzs\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.515302 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.515032    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-home\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.515302 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.515065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-dshm\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.515544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.515523    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-model-cache\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.515616 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.515524    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tmp-dir\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.515616 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.515555    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-home\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.517360 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.517340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-dshm\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.517731 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.517708    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tls-certs\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.523239 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.523210    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wjzs\" (UniqueName: \"kubernetes.io/projected/8b9ef65a-1d7f-438b-9503-cfb8b0963503-kube-api-access-5wjzs\") pod \"simulator-w4-03f258-kserve-69856cb965-lh979\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") " pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.565693 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.565648    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:34.839055 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:34.838990    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"]
Sep 15 22:41:34.842144 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:41:34.842116    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b9ef65a_1d7f_438b_9503_cfb8b0963503.slice/crio-54482be5753f76e8cea2617dd206c81ba2568587ce47eedcb75fa454e07930e0 WatchSource:0}: Error finding container 54482be5753f76e8cea2617dd206c81ba2568587ce47eedcb75fa454e07930e0: Status 404 returned error can't find the container with id 54482be5753f76e8cea2617dd206c81ba2568587ce47eedcb75fa454e07930e0
Sep 15 22:41:35.474638 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:35.474591    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979" event={"ID":"8b9ef65a-1d7f-438b-9503-cfb8b0963503","Type":"ContainerStarted","Data":"54482be5753f76e8cea2617dd206c81ba2568587ce47eedcb75fa454e07930e0"}
Sep 15 22:41:36.480912 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:36.480878    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979" event={"ID":"8b9ef65a-1d7f-438b-9503-cfb8b0963503","Type":"ContainerStarted","Data":"4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c"}
Sep 15 22:41:36.498331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:36.498273    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979" podStartSLOduration=1.396749481 podStartE2EDuration="2.498251904s" totalImagesPullingTime="1.101502423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:41:36.497787136 +0000 UTC m=+836.251411264" watchObservedRunningTime="2026-09-15 22:41:36.498251904 +0000 UTC m=+836.251876032"
Sep 15 22:41:44.566298 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:44.566255    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:44.566704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:44.566341    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:44.579052 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:44.579020    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:45.533731 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:45.533702    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:41:46.688718 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:46.688683    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"]
Sep 15 22:41:46.689281 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:46.689246    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy" containerID="cri-o://ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec" gracePeriod=30
Sep 15 22:41:47.414285 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.414251    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"]
Sep 15 22:41:47.414678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.414663    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy"
Sep 15 22:41:47.414678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.414679    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f1c8e3-d9ae-4854-9872-4dab7978bcd7" containerName="istio-proxy"
Sep 15 22:41:47.418273 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.418246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.420989 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.420962    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-93550aef-openshift-default-dockercfg-d9l27\""
Sep 15 22:41:47.431452 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.431420    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"]
Sep 15 22:41:47.443706 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.443668    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-token\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.443706 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.443708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-data\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.443940 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.443730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-socket\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.443940 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.443750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-certs\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.443940 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.443811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-credential-socket\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.443940 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.443859    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-podinfo\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.443940 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.443885    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzhkn\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-kube-api-access-dzhkn\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.444155 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.444009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88a3b6e4-dc15-42d7-a816-3d28a7180650-istiod-ca-cert\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.444155 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.444052    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-envoy\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.544837 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.544796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88a3b6e4-dc15-42d7-a816-3d28a7180650-istiod-ca-cert\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.545817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.545793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-envoy\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.546798 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.546253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-token\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.547453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.546992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-data\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.547649 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.547619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-socket\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.548171 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.548153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-certs\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.548583 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.547345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-data\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.548583 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.548005    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-socket\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.549129 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.549110    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-certs\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.549422 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.549406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-envoy\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.549573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.549558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-credential-socket\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.550125 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.549947    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-credential-socket\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.551572 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.551555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-podinfo\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.552388 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.552369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzhkn\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-kube-api-access-dzhkn\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.553750 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.553626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-podinfo\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.553978 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.553947    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88a3b6e4-dc15-42d7-a816-3d28a7180650-istiod-ca-cert\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.555263 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.555219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-token\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.563184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.563145    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzhkn\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-kube-api-access-dzhkn\") pod \"e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") " pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:47.733223 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:47.733182    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:48.011762 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:48.011723    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"]
Sep 15 22:41:48.013855 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:41:48.013825    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a3b6e4_dc15_42d7_a816_3d28a7180650.slice/crio-409ff04b412f98888ca9a1843829d01bf2796f2e9f7bd8c3f16a4c61f6d04990 WatchSource:0}: Error finding container 409ff04b412f98888ca9a1843829d01bf2796f2e9f7bd8c3f16a4c61f6d04990: Status 404 returned error can't find the container with id 409ff04b412f98888ca9a1843829d01bf2796f2e9f7bd8c3f16a4c61f6d04990
Sep 15 22:41:48.016279 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:48.016241    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:48.016383 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:48.016328    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:48.016383 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:48.016372    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:41:48.543861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:48.543822    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" event={"ID":"88a3b6e4-dc15-42d7-a816-3d28a7180650","Type":"ContainerStarted","Data":"b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6"}
Sep 15 22:41:48.544025 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:48.543887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" event={"ID":"88a3b6e4-dc15-42d7-a816-3d28a7180650","Type":"ContainerStarted","Data":"409ff04b412f98888ca9a1843829d01bf2796f2e9f7bd8c3f16a4c61f6d04990"}
Sep 15 22:41:48.570932 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:48.570869    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" podStartSLOduration=1.570848625 podStartE2EDuration="1.570848625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:41:48.56993973 +0000 UTC m=+848.323563867" watchObservedRunningTime="2026-09-15 22:41:48.570848625 +0000 UTC m=+848.324472751"
Sep 15 22:41:48.733554 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:48.733511    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:49.739109 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:49.739058    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:50.555947 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:50.555917    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:50.556919 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:50.556896    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:41:52.053618 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.053590    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.102795    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tmsc\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-kube-api-access-8tmsc\") pod \"623da280-afc2-49c1-848e-d1ecc9d232a4\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") "
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.102871    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-credential-socket\") pod \"623da280-afc2-49c1-848e-d1ecc9d232a4\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") "
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.102908    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-envoy\") pod \"623da280-afc2-49c1-848e-d1ecc9d232a4\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") "
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.102941    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623da280-afc2-49c1-848e-d1ecc9d232a4-istiod-ca-cert\") pod \"623da280-afc2-49c1-848e-d1ecc9d232a4\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") "
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.103007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-token\") pod \"623da280-afc2-49c1-848e-d1ecc9d232a4\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") "
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.103031    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-podinfo\") pod \"623da280-afc2-49c1-848e-d1ecc9d232a4\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") "
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.103055    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-data\") pod \"623da280-afc2-49c1-848e-d1ecc9d232a4\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") "
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.103080    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-certs\") pod \"623da280-afc2-49c1-848e-d1ecc9d232a4\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") "
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.103158    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-socket\") pod \"623da280-afc2-49c1-848e-d1ecc9d232a4\" (UID: \"623da280-afc2-49c1-848e-d1ecc9d232a4\") "
Sep 15 22:41:52.104640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.103815    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "623da280-afc2-49c1-848e-d1ecc9d232a4" (UID: "623da280-afc2-49c1-848e-d1ecc9d232a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:52.105340 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.104891    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "623da280-afc2-49c1-848e-d1ecc9d232a4" (UID: "623da280-afc2-49c1-848e-d1ecc9d232a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:52.108500 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.108455    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-data" (OuterVolumeSpecName: "istio-data") pod "623da280-afc2-49c1-848e-d1ecc9d232a4" (UID: "623da280-afc2-49c1-848e-d1ecc9d232a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:52.111600 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.111274    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-kube-api-access-8tmsc" (OuterVolumeSpecName: "kube-api-access-8tmsc") pod "623da280-afc2-49c1-848e-d1ecc9d232a4" (UID: "623da280-afc2-49c1-848e-d1ecc9d232a4"). InnerVolumeSpecName "kube-api-access-8tmsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:41:52.111908 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.111869    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "623da280-afc2-49c1-848e-d1ecc9d232a4" (UID: "623da280-afc2-49c1-848e-d1ecc9d232a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:52.112086 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.111934    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/623da280-afc2-49c1-848e-d1ecc9d232a4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "623da280-afc2-49c1-848e-d1ecc9d232a4" (UID: "623da280-afc2-49c1-848e-d1ecc9d232a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:41:52.113527 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.113491    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "623da280-afc2-49c1-848e-d1ecc9d232a4" (UID: "623da280-afc2-49c1-848e-d1ecc9d232a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:41:52.113527 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.113516    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-token" (OuterVolumeSpecName: "istio-token") pod "623da280-afc2-49c1-848e-d1ecc9d232a4" (UID: "623da280-afc2-49c1-848e-d1ecc9d232a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:41:52.113956 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.113936    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "623da280-afc2-49c1-848e-d1ecc9d232a4" (UID: "623da280-afc2-49c1-848e-d1ecc9d232a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:41:52.205015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.204974    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:52.205015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.205008    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:52.205015 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.205019    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:52.205294 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.205028    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:52.205294 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.205036    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:52.205294 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.205045    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tmsc\" (UniqueName: \"kubernetes.io/projected/623da280-afc2-49c1-848e-d1ecc9d232a4-kube-api-access-8tmsc\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:52.205294 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.205055    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:52.205294 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.205063    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623da280-afc2-49c1-848e-d1ecc9d232a4-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:52.205294 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.205070    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623da280-afc2-49c1-848e-d1ecc9d232a4-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:41:52.568779 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.568742    2578 generic.go:356] "Generic (PLEG): container finished" podID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerID="ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec" exitCode=0
Sep 15 22:41:52.568990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.568798    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"
Sep 15 22:41:52.568990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.568835    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" event={"ID":"623da280-afc2-49c1-848e-d1ecc9d232a4","Type":"ContainerDied","Data":"ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec"}
Sep 15 22:41:52.568990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.568938    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz" event={"ID":"623da280-afc2-49c1-848e-d1ecc9d232a4","Type":"ContainerDied","Data":"6ff137a1482548f4a346f514ef05e9935bad2c6adbff23fc6099f8c21a653749"}
Sep 15 22:41:52.568990 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.568971    2578 scope.go:117] "RemoveContainer" containerID="ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec"
Sep 15 22:41:52.580515 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.580491    2578 scope.go:117] "RemoveContainer" containerID="138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3"
Sep 15 22:41:52.589630 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.589604    2578 scope.go:117] "RemoveContainer" containerID="ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec"
Sep 15 22:41:52.590035 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:41:52.590014    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec\": container with ID starting with ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec not found: ID does not exist" containerID="ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec"
Sep 15 22:41:52.590158 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.590049    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec"} err="failed to get container status \"ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec\": rpc error: code = NotFound desc = could not find container \"ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec\": container with ID starting with ac5213fc68f2d576836b9de07a0b2cb377c99531e8335e4c048c5b0314907fec not found: ID does not exist"
Sep 15 22:41:52.590158 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.590075    2578 scope.go:117] "RemoveContainer" containerID="138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3"
Sep 15 22:41:52.590474 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:41:52.590451    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3\": container with ID starting with 138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3 not found: ID does not exist" containerID="138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3"
Sep 15 22:41:52.590553 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.590480    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3"} err="failed to get container status \"138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3\": rpc error: code = NotFound desc = could not find container \"138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3\": container with ID starting with 138ad402d348b5b434f527489c682539c8300c2cd9cd9a7438b27e9dcbcd6cb3 not found: ID does not exist"
Sep 15 22:41:52.596742 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.596711    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"]
Sep 15 22:41:52.602628 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.602599    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-36d266fd-gw-openshift-default-84df997bb4-mg8dz"]
Sep 15 22:41:52.763231 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:41:52.763196    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" path="/var/lib/kubelet/pods/623da280-afc2-49c1-848e-d1ecc9d232a4/volumes"
Sep 15 22:42:00.948441 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.948406    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"]
Sep 15 22:42:00.948913 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.948859    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy"
Sep 15 22:42:00.948913 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.948880    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy"
Sep 15 22:42:00.948986 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.948915    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy"
Sep 15 22:42:00.948986 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.948924    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy"
Sep 15 22:42:00.949051 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.949008    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy"
Sep 15 22:42:00.952782 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.952761    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:00.957131 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.957085    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-fe020d\"/\"simulator-w0-fe020d-kserve-self-signed-certs\""
Sep 15 22:42:00.957131 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.957087    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-fe020d\"/\"kube-root-ca.crt\""
Sep 15 22:42:00.957375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.957085    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-fe020d\"/\"default-dockercfg-t99pp\""
Sep 15 22:42:00.957375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.957245    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-fe020d\"/\"openshift-service-ca.crt\""
Sep 15 22:42:00.959938 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:00.959914    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"]
Sep 15 22:42:01.095287 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.095249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-dshm\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.095287 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.095289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-home\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.095538 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.095330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-model-cache\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.095538 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.095471    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-tmp-dir\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.095538 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.095517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tslpl\" (UniqueName: \"kubernetes.io/projected/3b155b8f-b07e-4801-984f-3609f6bbfc00-kube-api-access-tslpl\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.095658 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.095541    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b155b8f-b07e-4801-984f-3609f6bbfc00-tls-certs\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.197037 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.197000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-tmp-dir\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.197257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.197055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tslpl\" (UniqueName: \"kubernetes.io/projected/3b155b8f-b07e-4801-984f-3609f6bbfc00-kube-api-access-tslpl\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.197257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.197073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b155b8f-b07e-4801-984f-3609f6bbfc00-tls-certs\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.197257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.197119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-dshm\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.197257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.197142    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-home\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.197257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.197173    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-model-cache\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.197564 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.197543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-model-cache\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.197661 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.197639    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-home\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.197729 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.197712    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-tmp-dir\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.199454 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.199394    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-dshm\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.199725 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.199704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b155b8f-b07e-4801-984f-3609f6bbfc00-tls-certs\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.206472 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.206447    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tslpl\" (UniqueName: \"kubernetes.io/projected/3b155b8f-b07e-4801-984f-3609f6bbfc00-kube-api-access-tslpl\") pod \"simulator-w0-fe020d-kserve-8447fcb47b-mbfcz\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") " pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.263309 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.263272    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:01.542825 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.542675    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"]
Sep 15 22:42:01.545674 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:42:01.545644    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b155b8f_b07e_4801_984f_3609f6bbfc00.slice/crio-5420165c867131a3695c64e276c7e4dc092804850c9eead2d56332fe6fefe84f WatchSource:0}: Error finding container 5420165c867131a3695c64e276c7e4dc092804850c9eead2d56332fe6fefe84f: Status 404 returned error can't find the container with id 5420165c867131a3695c64e276c7e4dc092804850c9eead2d56332fe6fefe84f
Sep 15 22:42:01.615664 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:01.615456    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz" event={"ID":"3b155b8f-b07e-4801-984f-3609f6bbfc00","Type":"ContainerStarted","Data":"5420165c867131a3695c64e276c7e4dc092804850c9eead2d56332fe6fefe84f"}
Sep 15 22:42:02.629135 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:02.628587    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz" event={"ID":"3b155b8f-b07e-4801-984f-3609f6bbfc00","Type":"ContainerStarted","Data":"ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c"}
Sep 15 22:42:02.650070 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:02.650001    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz" podStartSLOduration=2.649974439 podStartE2EDuration="2.649974439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:42:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:42:02.648404742 +0000 UTC m=+862.402028871" watchObservedRunningTime="2026-09-15 22:42:02.649974439 +0000 UTC m=+862.403598570"
Sep 15 22:42:11.263924 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:11.263873    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:11.264367 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:11.263950    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:11.277077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:11.277049    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:11.684905 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:11.684817    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:42:13.685075 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:13.685042    2578 generic.go:356] "Generic (PLEG): container finished" podID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerID="b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6" exitCode=0
Sep 15 22:42:13.685545 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:13.685116    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" event={"ID":"88a3b6e4-dc15-42d7-a816-3d28a7180650","Type":"ContainerDied","Data":"b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6"}
Sep 15 22:42:13.685590 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:13.685578    2578 scope.go:117] "RemoveContainer" containerID="b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6"
Sep 15 22:42:13.686789 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:13.686754    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:42:13.686971 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:13.686843    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:42:13.686971 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:13.686921    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:42:14.692317 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:14.692280    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" event={"ID":"88a3b6e4-dc15-42d7-a816-3d28a7180650","Type":"ContainerStarted","Data":"bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14"}
Sep 15 22:42:14.733881 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:14.733838    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:42:14.734107 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:14.734063    2578 patch_prober.go:28] interesting pod/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 22:42:14.734207 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:14.734153    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 22:42:15.734401 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:15.734354    2578 patch_prober.go:28] interesting pod/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 22:42:15.734943 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:15.734433    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 22:42:16.733611 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:16.733570    2578 patch_prober.go:28] interesting pod/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 22:42:16.733777 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:16.733629    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 22:42:17.733581 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:17.733542    2578 patch_prober.go:28] interesting pod/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 22:42:17.733984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:17.733602    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 22:42:18.734304 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:18.734264    2578 patch_prober.go:28] interesting pod/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 22:42:18.734697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:18.734322    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 22:42:19.734467 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:19.734426    2578 patch_prober.go:28] interesting pod/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 22:42:19.734876 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:19.734514    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 22:42:20.556068 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:20.556029    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:42:20.738705 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:20.738675    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:42:20.739657 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:20.739634    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:42:23.094689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:23.094652    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"]
Sep 15 22:42:23.095125 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:23.094909    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy" containerID="cri-o://bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14" gracePeriod=30
Sep 15 22:42:28.354274 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.354241    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:42:28.472152 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472075    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzhkn\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-kube-api-access-dzhkn\") pod \"88a3b6e4-dc15-42d7-a816-3d28a7180650\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") "
Sep 15 22:42:28.472332 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472165    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-podinfo\") pod \"88a3b6e4-dc15-42d7-a816-3d28a7180650\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") "
Sep 15 22:42:28.472332 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472232    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88a3b6e4-dc15-42d7-a816-3d28a7180650-istiod-ca-cert\") pod \"88a3b6e4-dc15-42d7-a816-3d28a7180650\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") "
Sep 15 22:42:28.472332 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472258    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-envoy\") pod \"88a3b6e4-dc15-42d7-a816-3d28a7180650\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") "
Sep 15 22:42:28.472332 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472291    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-data\") pod \"88a3b6e4-dc15-42d7-a816-3d28a7180650\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") "
Sep 15 22:42:28.472332 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472334    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-credential-socket\") pod \"88a3b6e4-dc15-42d7-a816-3d28a7180650\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") "
Sep 15 22:42:28.472578 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472367    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-socket\") pod \"88a3b6e4-dc15-42d7-a816-3d28a7180650\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") "
Sep 15 22:42:28.472578 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472397    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-certs\") pod \"88a3b6e4-dc15-42d7-a816-3d28a7180650\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") "
Sep 15 22:42:28.472578 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472430    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-token\") pod \"88a3b6e4-dc15-42d7-a816-3d28a7180650\" (UID: \"88a3b6e4-dc15-42d7-a816-3d28a7180650\") "
Sep 15 22:42:28.472739 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472601    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-data" (OuterVolumeSpecName: "istio-data") pod "88a3b6e4-dc15-42d7-a816-3d28a7180650" (UID: "88a3b6e4-dc15-42d7-a816-3d28a7180650"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:42:28.472739 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472705    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88a3b6e4-dc15-42d7-a816-3d28a7180650-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "88a3b6e4-dc15-42d7-a816-3d28a7180650" (UID: "88a3b6e4-dc15-42d7-a816-3d28a7180650"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:42:28.472862 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472782    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88a3b6e4-dc15-42d7-a816-3d28a7180650-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:42:28.472862 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472796    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:42:28.472964 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.472943    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "88a3b6e4-dc15-42d7-a816-3d28a7180650" (UID: "88a3b6e4-dc15-42d7-a816-3d28a7180650"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:42:28.473175 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.473145    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "88a3b6e4-dc15-42d7-a816-3d28a7180650" (UID: "88a3b6e4-dc15-42d7-a816-3d28a7180650"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:42:28.473175 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.473158    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "88a3b6e4-dc15-42d7-a816-3d28a7180650" (UID: "88a3b6e4-dc15-42d7-a816-3d28a7180650"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:42:28.474538 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.474516    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "88a3b6e4-dc15-42d7-a816-3d28a7180650" (UID: "88a3b6e4-dc15-42d7-a816-3d28a7180650"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:42:28.474631 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.474576    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-token" (OuterVolumeSpecName: "istio-token") pod "88a3b6e4-dc15-42d7-a816-3d28a7180650" (UID: "88a3b6e4-dc15-42d7-a816-3d28a7180650"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:42:28.474631 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.474593    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-kube-api-access-dzhkn" (OuterVolumeSpecName: "kube-api-access-dzhkn") pod "88a3b6e4-dc15-42d7-a816-3d28a7180650" (UID: "88a3b6e4-dc15-42d7-a816-3d28a7180650"). InnerVolumeSpecName "kube-api-access-dzhkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:42:28.474790 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.474776    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "88a3b6e4-dc15-42d7-a816-3d28a7180650" (UID: "88a3b6e4-dc15-42d7-a816-3d28a7180650"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:42:28.573671 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.573634    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:42:28.573671 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.573667    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:42:28.573671 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.573677    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:42:28.573902 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.573685    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:42:28.573902 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.573694    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzhkn\" (UniqueName: \"kubernetes.io/projected/88a3b6e4-dc15-42d7-a816-3d28a7180650-kube-api-access-dzhkn\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:42:28.573902 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.573704    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:42:28.573902 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.573714    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88a3b6e4-dc15-42d7-a816-3d28a7180650-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:42:28.779367 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.779333    2578 generic.go:356] "Generic (PLEG): container finished" podID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerID="bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14" exitCode=0
Sep 15 22:42:28.779535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.779378    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"
Sep 15 22:42:28.779535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.779389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" event={"ID":"88a3b6e4-dc15-42d7-a816-3d28a7180650","Type":"ContainerDied","Data":"bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14"}
Sep 15 22:42:28.779535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.779430    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv" event={"ID":"88a3b6e4-dc15-42d7-a816-3d28a7180650","Type":"ContainerDied","Data":"409ff04b412f98888ca9a1843829d01bf2796f2e9f7bd8c3f16a4c61f6d04990"}
Sep 15 22:42:28.779535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.779451    2578 scope.go:117] "RemoveContainer" containerID="bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14"
Sep 15 22:42:28.788693 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.788675    2578 scope.go:117] "RemoveContainer" containerID="b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6"
Sep 15 22:42:28.800507 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.800484    2578 scope.go:117] "RemoveContainer" containerID="bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14"
Sep 15 22:42:28.800852 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:42:28.800833    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14\": container with ID starting with bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14 not found: ID does not exist" containerID="bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14"
Sep 15 22:42:28.800927 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.800861    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14"} err="failed to get container status \"bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14\": rpc error: code = NotFound desc = could not find container \"bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14\": container with ID starting with bf2ac5b667b8879acb44a7ca463b6e75e8aee4d4bb4caa0be2ec4aad2b1f0a14 not found: ID does not exist"
Sep 15 22:42:28.800927 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.800882    2578 scope.go:117] "RemoveContainer" containerID="b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6"
Sep 15 22:42:28.801217 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:42:28.801199    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6\": container with ID starting with b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6 not found: ID does not exist" containerID="b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6"
Sep 15 22:42:28.801298 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.801221    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6"} err="failed to get container status \"b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6\": rpc error: code = NotFound desc = could not find container \"b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6\": container with ID starting with b065f2c844437001abb081b379c940a58a9fe6bacac2a2363165867d544de2d6 not found: ID does not exist"
Sep 15 22:42:28.801409 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.801390    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"]
Sep 15 22:42:28.806354 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:28.806323    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-93550aef-openshift-default-7b499975cb-4bbcv"]
Sep 15 22:42:30.762743 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:30.762705    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" path="/var/lib/kubelet/pods/88a3b6e4-dc15-42d7-a816-3d28a7180650/volumes"
Sep 15 22:42:48.873959 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.873926    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"]
Sep 15 22:42:48.874382 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.874315    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy"
Sep 15 22:42:48.874382 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.874327    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy"
Sep 15 22:42:48.874382 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.874335    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy"
Sep 15 22:42:48.874382 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.874341    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy"
Sep 15 22:42:48.874533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.874398    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="623da280-afc2-49c1-848e-d1ecc9d232a4" containerName="istio-proxy"
Sep 15 22:42:48.874533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.874409    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy"
Sep 15 22:42:48.874533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.874416    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a3b6e4-dc15-42d7-a816-3d28a7180650" containerName="istio-proxy"
Sep 15 22:42:48.876904 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.876887    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:48.880073 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.880043    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fffdae\"/\"openshift-service-ca.crt\""
Sep 15 22:42:48.880073 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.880043    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fffdae\"/\"default-dockercfg-56crm\""
Sep 15 22:42:48.881467 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.881451    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fffdae\"/\"kube-root-ca.crt\""
Sep 15 22:42:48.881593 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.881566    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fffdae\"/\"test-model-fffdae-kserve-self-signed-certs\""
Sep 15 22:42:48.887834 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.887805    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"]
Sep 15 22:42:48.973303 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.973260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-model-cache\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:48.973515 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.973331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-988th\" (UniqueName: \"kubernetes.io/projected/c50aa469-ded4-4043-b58e-450ab73f2ac4-kube-api-access-988th\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:48.973515 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.973456    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-home\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:48.973515 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.973501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-dshm\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:48.973646 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.973567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c50aa469-ded4-4043-b58e-450ab73f2ac4-tls-certs\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:48.973646 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:48.973619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-tmp-dir\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.074338 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.074298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-988th\" (UniqueName: \"kubernetes.io/projected/c50aa469-ded4-4043-b58e-450ab73f2ac4-kube-api-access-988th\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.074544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.074363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-home\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.074544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.074390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-dshm\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.074544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.074413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c50aa469-ded4-4043-b58e-450ab73f2ac4-tls-certs\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.074544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.074444    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-tmp-dir\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.074544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.074485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-model-cache\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.074844 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.074820    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-tmp-dir\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.074930 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.074826    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-home\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.074930 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.074888    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-model-cache\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.076757 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.076735    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-dshm\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.077087 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.077071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c50aa469-ded4-4043-b58e-450ab73f2ac4-tls-certs\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.083518 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.083489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-988th\" (UniqueName: \"kubernetes.io/projected/c50aa469-ded4-4043-b58e-450ab73f2ac4-kube-api-access-988th\") pod \"test-model-fffdae-kserve-86499d5bcc-r8pqm\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") " pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.188571 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.188538    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:49.458789 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.458529    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"]
Sep 15 22:42:49.461293 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:42:49.461262    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc50aa469_ded4_4043_b58e_450ab73f2ac4.slice/crio-afbaa160c37a557985a7a4d547df16973d7f0863b8aa074d3ce3b243090b0077 WatchSource:0}: Error finding container afbaa160c37a557985a7a4d547df16973d7f0863b8aa074d3ce3b243090b0077: Status 404 returned error can't find the container with id afbaa160c37a557985a7a4d547df16973d7f0863b8aa074d3ce3b243090b0077
Sep 15 22:42:49.890214 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.890125    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm" event={"ID":"c50aa469-ded4-4043-b58e-450ab73f2ac4","Type":"ContainerStarted","Data":"31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90"}
Sep 15 22:42:49.890214 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.890169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm" event={"ID":"c50aa469-ded4-4043-b58e-450ab73f2ac4","Type":"ContainerStarted","Data":"afbaa160c37a557985a7a4d547df16973d7f0863b8aa074d3ce3b243090b0077"}
Sep 15 22:42:49.909718 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:49.909663    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm" podStartSLOduration=1.90964432 podStartE2EDuration="1.90964432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:42:49.908177138 +0000 UTC m=+909.661801265" watchObservedRunningTime="2026-09-15 22:42:49.90964432 +0000 UTC m=+909.663268447"
Sep 15 22:42:59.188743 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:59.188702    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:59.188743 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:59.188755    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:59.201382 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:59.201348    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:42:59.951642 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:42:59.951611    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:43:56.073943 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:43:56.073903    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"]
Sep 15 22:43:56.074545 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:43:56.074187    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy" containerID="cri-o://2e2200f58083f837331078abf3bb358a8f97ec77756bb68e40cc611b991f6072" gracePeriod=30
Sep 15 22:44:01.235157 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.235121    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" event={"ID":"00bcdafe-87c7-4c02-8a26-21ef6add9c79","Type":"ContainerDied","Data":"2e2200f58083f837331078abf3bb358a8f97ec77756bb68e40cc611b991f6072"}
Sep 15 22:44:01.235595 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.235188    2578 scope.go:117] "RemoveContainer" containerID="3e4230da6b07dfb84ecc6c3e6af24e5c55e60ad44c502af12fa805849eb69be4"
Sep 15 22:44:01.235595 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.235120    2578 generic.go:356] "Generic (PLEG): container finished" podID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerID="2e2200f58083f837331078abf3bb358a8f97ec77756bb68e40cc611b991f6072" exitCode=0
Sep 15 22:44:01.333881 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.333851    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:44:01.504838 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.504794    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-data\") pod \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") "
Sep 15 22:44:01.505045 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.504862    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rks2\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-kube-api-access-9rks2\") pod \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") "
Sep 15 22:44:01.505045 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.504895    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-credential-socket\") pod \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") "
Sep 15 22:44:01.505045 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.504965    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-token\") pod \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") "
Sep 15 22:44:01.505045 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-envoy\") pod \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") "
Sep 15 22:44:01.505045 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505031    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-certs\") pod \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") "
Sep 15 22:44:01.505347 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505082    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istiod-ca-cert\") pod \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") "
Sep 15 22:44:01.505347 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505135    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-socket\") pod \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") "
Sep 15 22:44:01.505347 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505147    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-data" (OuterVolumeSpecName: "istio-data") pod "00bcdafe-87c7-4c02-8a26-21ef6add9c79" (UID: "00bcdafe-87c7-4c02-8a26-21ef6add9c79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:44:01.505347 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505163    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-podinfo\") pod \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\" (UID: \"00bcdafe-87c7-4c02-8a26-21ef6add9c79\") "
Sep 15 22:44:01.505562 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505386    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "00bcdafe-87c7-4c02-8a26-21ef6add9c79" (UID: "00bcdafe-87c7-4c02-8a26-21ef6add9c79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:44:01.505622 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505563    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "00bcdafe-87c7-4c02-8a26-21ef6add9c79" (UID: "00bcdafe-87c7-4c02-8a26-21ef6add9c79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:44:01.505622 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505597    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "00bcdafe-87c7-4c02-8a26-21ef6add9c79" (UID: "00bcdafe-87c7-4c02-8a26-21ef6add9c79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:44:01.505731 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505663    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "00bcdafe-87c7-4c02-8a26-21ef6add9c79" (UID: "00bcdafe-87c7-4c02-8a26-21ef6add9c79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:44:01.505731 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505692    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:44:01.505731 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505710    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:44:01.505731 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505723    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:44:01.505943 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.505737    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:44:01.507369 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.507335    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "00bcdafe-87c7-4c02-8a26-21ef6add9c79" (UID: "00bcdafe-87c7-4c02-8a26-21ef6add9c79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:44:01.507480 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.507391    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-kube-api-access-9rks2" (OuterVolumeSpecName: "kube-api-access-9rks2") pod "00bcdafe-87c7-4c02-8a26-21ef6add9c79" (UID: "00bcdafe-87c7-4c02-8a26-21ef6add9c79"). InnerVolumeSpecName "kube-api-access-9rks2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:44:01.507480 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.507392    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-token" (OuterVolumeSpecName: "istio-token") pod "00bcdafe-87c7-4c02-8a26-21ef6add9c79" (UID: "00bcdafe-87c7-4c02-8a26-21ef6add9c79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:44:01.507480 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.507416    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "00bcdafe-87c7-4c02-8a26-21ef6add9c79" (UID: "00bcdafe-87c7-4c02-8a26-21ef6add9c79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:44:01.606638 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.606598    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:44:01.606638 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.606628    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:44:01.606638 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.606637    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bcdafe-87c7-4c02-8a26-21ef6add9c79-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:44:01.606638 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.606647    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rks2\" (UniqueName: \"kubernetes.io/projected/00bcdafe-87c7-4c02-8a26-21ef6add9c79-kube-api-access-9rks2\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:44:01.606910 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:01.606656    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bcdafe-87c7-4c02-8a26-21ef6add9c79-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:44:02.241156 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:02.241126    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"
Sep 15 22:44:02.241622 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:02.241125    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2" event={"ID":"00bcdafe-87c7-4c02-8a26-21ef6add9c79","Type":"ContainerDied","Data":"d67a474913260a632a1411609b01b3c10b9129d53cd07542d2c3447fd31132c6"}
Sep 15 22:44:02.241622 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:02.241254    2578 scope.go:117] "RemoveContainer" containerID="2e2200f58083f837331078abf3bb358a8f97ec77756bb68e40cc611b991f6072"
Sep 15 22:44:02.264552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:02.264508    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"]
Sep 15 22:44:02.268589 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:02.268560    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-b08870-openshift-default-5559b9969b-czsd2"]
Sep 15 22:44:02.762908 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:02.762867    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" path="/var/lib/kubelet/pods/00bcdafe-87c7-4c02-8a26-21ef6add9c79/volumes"
Sep 15 22:44:36.661135 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.661086    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"]
Sep 15 22:44:36.661576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.661466    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy"
Sep 15 22:44:36.661576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.661477    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy"
Sep 15 22:44:36.661576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.661543    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy"
Sep 15 22:44:36.661576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.661552    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy"
Sep 15 22:44:36.664419 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.664400    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.667932 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.667908    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-fe020d\"/\"premium-w0-fe020d-kserve-self-signed-certs\""
Sep 15 22:44:36.694586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.694558    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"]
Sep 15 22:44:36.740286 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.740251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tls-certs\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.740476 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.740297    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh6l7\" (UniqueName: \"kubernetes.io/projected/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-kube-api-access-zh6l7\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.740476 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.740387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-home\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.740476 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.740421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-model-cache\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.740476 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.740455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-dshm\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.740620 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.740489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tmp-dir\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.841679 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.841640    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tls-certs\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.841874 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.841696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh6l7\" (UniqueName: \"kubernetes.io/projected/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-kube-api-access-zh6l7\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.841874 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.841743    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-home\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.841874 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.841761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-model-cache\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.841874 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.841798    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-dshm\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.841874 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.841828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tmp-dir\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.842241 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.842216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-home\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.842325 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.842269    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-model-cache\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.842325 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.842278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tmp-dir\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.844078 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.844056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-dshm\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.844275 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.844259    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tls-certs\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.855388 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.855318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh6l7\" (UniqueName: \"kubernetes.io/projected/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-kube-api-access-zh6l7\") pod \"premium-w0-fe020d-kserve-7549c5488f-w9rcb\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") " pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:36.976817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:36.976777    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:37.252608 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:37.252580    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"]
Sep 15 22:44:37.254730 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:44:37.254700    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ff99328_e4a2_4eb8_aa27_bd78fa34bc5f.slice/crio-be0dc3ae44634afbfb11aa1deb1672edd2193382b9118f673eaf9b89f56c3643 WatchSource:0}: Error finding container be0dc3ae44634afbfb11aa1deb1672edd2193382b9118f673eaf9b89f56c3643: Status 404 returned error can't find the container with id be0dc3ae44634afbfb11aa1deb1672edd2193382b9118f673eaf9b89f56c3643
Sep 15 22:44:37.414056 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:37.414009    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb" event={"ID":"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f","Type":"ContainerStarted","Data":"4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668"}
Sep 15 22:44:37.414056 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:37.414062    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb" event={"ID":"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f","Type":"ContainerStarted","Data":"be0dc3ae44634afbfb11aa1deb1672edd2193382b9118f673eaf9b89f56c3643"}
Sep 15 22:44:37.433573 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:37.433510    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb" podStartSLOduration=1.433491791 podStartE2EDuration="1.433491791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:44:37.43135821 +0000 UTC m=+1017.184982338" watchObservedRunningTime="2026-09-15 22:44:37.433491791 +0000 UTC m=+1017.187115917"
Sep 15 22:44:43.357118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.357053    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"]
Sep 15 22:44:43.357529 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.357512    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy"
Sep 15 22:44:43.357579 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.357531    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="00bcdafe-87c7-4c02-8a26-21ef6add9c79" containerName="istio-proxy"
Sep 15 22:44:43.362173 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.362150    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.364941 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.364917    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2bbfe736-openshift-default-dockercfg-sg6mp\""
Sep 15 22:44:43.380875 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.380842    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"]
Sep 15 22:44:43.510104 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.510059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvpdx\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-kube-api-access-vvpdx\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.510323 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.510165    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-data\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.510323 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.510211    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-credential-socket\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.510323 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.510234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-token\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.510323 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.510260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-podinfo\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.510323 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.510281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-certs\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.510323 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.510318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182f5bf6-5bc8-413a-beb4-de67f156251a-istiod-ca-cert\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.510586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.510352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-socket\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.510586 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.510387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-envoy\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611171 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-envoy\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611171 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611087    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvpdx\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-kube-api-access-vvpdx\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611180    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-data\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611222    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-credential-socket\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-token\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-podinfo\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-certs\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611410 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611367    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182f5bf6-5bc8-413a-beb4-de67f156251a-istiod-ca-cert\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611712 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-socket\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611712 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611610    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-data\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611712 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611639    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-credential-socket\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-socket\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.611871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.611831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-certs\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.612207 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.612184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182f5bf6-5bc8-413a-beb4-de67f156251a-istiod-ca-cert\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.613435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.613414    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-envoy\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.613654 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.613636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-podinfo\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.620783 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.620759    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-token\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.620999 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.620974    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvpdx\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-kube-api-access-vvpdx\") pod \"e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") " pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:43.679877 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:43.679839    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:44.020085 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:44:44.020050    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182f5bf6_5bc8_413a_beb4_de67f156251a.slice/crio-df70729726b22a41685ea3bac256259ea1f84440bc62854581c94be8d0e36967 WatchSource:0}: Error finding container df70729726b22a41685ea3bac256259ea1f84440bc62854581c94be8d0e36967: Status 404 returned error can't find the container with id df70729726b22a41685ea3bac256259ea1f84440bc62854581c94be8d0e36967
Sep 15 22:44:44.020891 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:44.020867    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"]
Sep 15 22:44:44.022657 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:44.022619    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:44:44.022739 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:44.022696    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:44:44.022739 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:44.022726    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:44:44.449694 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:44.449662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" event={"ID":"182f5bf6-5bc8-413a-beb4-de67f156251a","Type":"ContainerStarted","Data":"e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399"}
Sep 15 22:44:44.450084 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:44.449713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" event={"ID":"182f5bf6-5bc8-413a-beb4-de67f156251a","Type":"ContainerStarted","Data":"df70729726b22a41685ea3bac256259ea1f84440bc62854581c94be8d0e36967"}
Sep 15 22:44:44.473105 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:44.473036    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" podStartSLOduration=1.473017214 podStartE2EDuration="1.473017214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:44:44.470889869 +0000 UTC m=+1024.224513997" watchObservedRunningTime="2026-09-15 22:44:44.473017214 +0000 UTC m=+1024.226641341"
Sep 15 22:44:44.680572 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:44.680538    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:45.686847 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:45.686812    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:46.462174 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:46.462134    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:46.463335 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:46.463306    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:44:46.977770 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:46.977722    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:46.978254 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:46.977801    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:46.997110 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:46.997056    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:47.486500 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:47.486469    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:44:53.033508 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.033470    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"]
Sep 15 22:44:53.037720 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.037695    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.040356 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.040328    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2bbfe736-serving-cert\""
Sep 15 22:44:53.050114 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.050063    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"]
Sep 15 22:44:53.101246 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.101206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45e2df6-27fc-4f77-9881-ee93d1f23346-openshift-service-ca\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.101442 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.101324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-maas-api-tls\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.101442 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.101402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfs4k\" (UniqueName: \"kubernetes.io/projected/e45e2df6-27fc-4f77-9881-ee93d1f23346-kube-api-access-rfs4k\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.101442 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.101438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-metrics-certs\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.202512 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.202470    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-metrics-certs\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.202709 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.202553    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45e2df6-27fc-4f77-9881-ee93d1f23346-openshift-service-ca\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.202709 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.202603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-maas-api-tls\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.202709 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.202654    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfs4k\" (UniqueName: \"kubernetes.io/projected/e45e2df6-27fc-4f77-9881-ee93d1f23346-kube-api-access-rfs4k\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.202820 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:44:53.202779    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-serving-cert: secret "maas-api-e2e-mt-2bbfe736-serving-cert" not found
Sep 15 22:44:53.202873 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:44:53.202860    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-maas-api-tls podName:e45e2df6-27fc-4f77-9881-ee93d1f23346 nodeName:}" failed. No retries permitted until 2026-09-15 22:44:53.702837642 +0000 UTC m=+1033.456461751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-maas-api-tls") pod "maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66" (UID: "e45e2df6-27fc-4f77-9881-ee93d1f23346") : secret "maas-api-e2e-mt-2bbfe736-serving-cert" not found
Sep 15 22:44:53.203339 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.203318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45e2df6-27fc-4f77-9881-ee93d1f23346-openshift-service-ca\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.205078 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.205053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-metrics-certs\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.212003 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.211973    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfs4k\" (UniqueName: \"kubernetes.io/projected/e45e2df6-27fc-4f77-9881-ee93d1f23346-kube-api-access-rfs4k\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.708585 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.708539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-maas-api-tls\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.711303 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.711277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-maas-api-tls\") pod \"maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:53.951330 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:53.951287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:54.250540 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:54.250467    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"]
Sep 15 22:44:54.252698 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:44:54.252669    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45e2df6_27fc_4f77_9881_ee93d1f23346.slice/crio-e5b78b23e01eb50b470839b6336e85863cc9687a745e9cd3cb7b0ae94e69dbb4 WatchSource:0}: Error finding container e5b78b23e01eb50b470839b6336e85863cc9687a745e9cd3cb7b0ae94e69dbb4: Status 404 returned error can't find the container with id e5b78b23e01eb50b470839b6336e85863cc9687a745e9cd3cb7b0ae94e69dbb4
Sep 15 22:44:54.501718 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:54.501628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66" event={"ID":"e45e2df6-27fc-4f77-9881-ee93d1f23346","Type":"ContainerStarted","Data":"e5b78b23e01eb50b470839b6336e85863cc9687a745e9cd3cb7b0ae94e69dbb4"}
Sep 15 22:44:55.507758 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:55.507718    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66" event={"ID":"e45e2df6-27fc-4f77-9881-ee93d1f23346","Type":"ContainerStarted","Data":"4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e"}
Sep 15 22:44:55.508210 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:55.507820    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:44:55.526548 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:44:55.526488    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66" podStartSLOduration=2.154736949 podStartE2EDuration="2.526471206s" totalImagesPullingTime="371.734257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:44:55.525459727 +0000 UTC m=+1035.279083856" watchObservedRunningTime="2026-09-15 22:44:55.526471206 +0000 UTC m=+1035.280095349"
Sep 15 22:45:00.143012 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:00.142977    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g"]
Sep 15 22:45:00.147146 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:00.147125    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g"
Sep 15 22:45:00.149615 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:00.149593    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hnj7x\""
Sep 15 22:45:00.158203 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:00.158173    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g"]
Sep 15 22:45:00.159958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:00.159930    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g"
Sep 15 22:45:00.437479 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:00.437453    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g"]
Sep 15 22:45:00.440390 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:45:00.440360    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f480852_191c_4f26_af93_622acfa750fe.slice/crio-9f2ce9c792e87d5e41aa04759848c450cfe524146ea9bcbf0e9529c21aa7c9b3 WatchSource:0}: Error finding container 9f2ce9c792e87d5e41aa04759848c450cfe524146ea9bcbf0e9529c21aa7c9b3: Status 404 returned error can't find the container with id 9f2ce9c792e87d5e41aa04759848c450cfe524146ea9bcbf0e9529c21aa7c9b3
Sep 15 22:45:00.533973 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:00.533936    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g" event={"ID":"1f480852-191c-4f26-af93-622acfa750fe","Type":"ContainerStarted","Data":"9f2ce9c792e87d5e41aa04759848c450cfe524146ea9bcbf0e9529c21aa7c9b3"}
Sep 15 22:45:01.517432 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:01.517404    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:45:03.552427 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:03.552376    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g" event={"ID":"1f480852-191c-4f26-af93-622acfa750fe","Type":"ContainerStarted","Data":"2e75453b259fda1f249595e0de48695670ba5fbbbfa7415181e23db2382489e7"}
Sep 15 22:45:03.570215 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:03.570157    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g" podStartSLOduration=0.89027592 podStartE2EDuration="3.570140373s" totalImagesPullingTime="2.679864453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:45:03.567908914 +0000 UTC m=+1043.321533043" watchObservedRunningTime="2026-09-15 22:45:03.570140373 +0000 UTC m=+1043.323764499"
Sep 15 22:45:04.559627 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:04.559594    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f480852-191c-4f26-af93-622acfa750fe" containerID="2e75453b259fda1f249595e0de48695670ba5fbbbfa7415181e23db2382489e7" exitCode=0
Sep 15 22:45:04.560126 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:04.559650    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g" event={"ID":"1f480852-191c-4f26-af93-622acfa750fe","Type":"ContainerDied","Data":"2e75453b259fda1f249595e0de48695670ba5fbbbfa7415181e23db2382489e7"}
Sep 15 22:45:05.745503 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:05.745474    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g"
Sep 15 22:45:06.571749 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:06.571714    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g"
Sep 15 22:45:06.571749 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:06.571737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825205-s5l2g" event={"ID":"1f480852-191c-4f26-af93-622acfa750fe","Type":"ContainerDied","Data":"9f2ce9c792e87d5e41aa04759848c450cfe524146ea9bcbf0e9529c21aa7c9b3"}
Sep 15 22:45:06.571989 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:06.571790    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f2ce9c792e87d5e41aa04759848c450cfe524146ea9bcbf0e9529c21aa7c9b3"
Sep 15 22:45:14.616643 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:14.616606    2578 generic.go:356] "Generic (PLEG): container finished" podID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerID="e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399" exitCode=0
Sep 15 22:45:14.617079 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:14.616678    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" event={"ID":"182f5bf6-5bc8-413a-beb4-de67f156251a","Type":"ContainerDied","Data":"e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399"}
Sep 15 22:45:14.617158 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:14.617120    2578 scope.go:117] "RemoveContainer" containerID="e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399"
Sep 15 22:45:14.618370 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:14.618337    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:45:14.618463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:14.618412    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:45:14.618463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:14.618446    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:45:15.623911 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:15.623872    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" event={"ID":"182f5bf6-5bc8-413a-beb4-de67f156251a","Type":"ContainerStarted","Data":"1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b"}
Sep 15 22:45:15.680302 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:15.680259    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:45:15.681351 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:15.681327    2578 patch_prober.go:28] interesting pod/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 22:45:15.681412 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:15.681388    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 22:45:16.462506 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:16.462471    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:45:16.680704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:16.680664    2578 patch_prober.go:28] interesting pod/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 22:45:16.681080 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:16.680726    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 22:45:17.680462 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:17.680419    2578 patch_prober.go:28] interesting pod/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 22:45:17.680678 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:17.680491    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 22:45:18.680407 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:18.680365    2578 patch_prober.go:28] interesting pod/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 22:45:18.680793 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:18.680432    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 22:45:19.680698 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:19.680659    2578 patch_prober.go:28] interesting pod/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 22:45:19.681210 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:19.680712    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 22:45:20.680568 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:20.680526    2578 patch_prober.go:28] interesting pod/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 22:45:20.680756 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:20.680586    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 22:45:21.684820 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:21.684783    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:45:21.685955 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:21.685931    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:45:36.970214 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:36.970175    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"]
Sep 15 22:45:36.970779 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:36.970494    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66" podUID="e45e2df6-27fc-4f77-9881-ee93d1f23346" containerName="maas-api" containerID="cri-o://4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e" gracePeriod=30
Sep 15 22:45:37.229110 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.229023    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:45:37.256292 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.256261    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-maas-api-tls\") pod \"e45e2df6-27fc-4f77-9881-ee93d1f23346\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") "
Sep 15 22:45:37.256483 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.256306    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfs4k\" (UniqueName: \"kubernetes.io/projected/e45e2df6-27fc-4f77-9881-ee93d1f23346-kube-api-access-rfs4k\") pod \"e45e2df6-27fc-4f77-9881-ee93d1f23346\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") "
Sep 15 22:45:37.256483 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.256373    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-metrics-certs\") pod \"e45e2df6-27fc-4f77-9881-ee93d1f23346\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") "
Sep 15 22:45:37.256483 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.256429    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45e2df6-27fc-4f77-9881-ee93d1f23346-openshift-service-ca\") pod \"e45e2df6-27fc-4f77-9881-ee93d1f23346\" (UID: \"e45e2df6-27fc-4f77-9881-ee93d1f23346\") "
Sep 15 22:45:37.256818 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.256791    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e45e2df6-27fc-4f77-9881-ee93d1f23346-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e45e2df6-27fc-4f77-9881-ee93d1f23346" (UID: "e45e2df6-27fc-4f77-9881-ee93d1f23346"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:45:37.259164 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.259134    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45e2df6-27fc-4f77-9881-ee93d1f23346-kube-api-access-rfs4k" (OuterVolumeSpecName: "kube-api-access-rfs4k") pod "e45e2df6-27fc-4f77-9881-ee93d1f23346" (UID: "e45e2df6-27fc-4f77-9881-ee93d1f23346"). InnerVolumeSpecName "kube-api-access-rfs4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:45:37.259316 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.259171    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e45e2df6-27fc-4f77-9881-ee93d1f23346" (UID: "e45e2df6-27fc-4f77-9881-ee93d1f23346"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:45:37.259511 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.259484    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e45e2df6-27fc-4f77-9881-ee93d1f23346" (UID: "e45e2df6-27fc-4f77-9881-ee93d1f23346"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:45:37.360219 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.360176    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-maas-api-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:37.360219 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.360219    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfs4k\" (UniqueName: \"kubernetes.io/projected/e45e2df6-27fc-4f77-9881-ee93d1f23346-kube-api-access-rfs4k\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:37.360497 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.360232    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45e2df6-27fc-4f77-9881-ee93d1f23346-metrics-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:37.360497 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.360245    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45e2df6-27fc-4f77-9881-ee93d1f23346-openshift-service-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:37.740112 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.740052    2578 generic.go:356] "Generic (PLEG): container finished" podID="e45e2df6-27fc-4f77-9881-ee93d1f23346" containerID="4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e" exitCode=0
Sep 15 22:45:37.740289 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.740240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66" event={"ID":"e45e2df6-27fc-4f77-9881-ee93d1f23346","Type":"ContainerDied","Data":"4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e"}
Sep 15 22:45:37.740352 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.740337    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66" event={"ID":"e45e2df6-27fc-4f77-9881-ee93d1f23346","Type":"ContainerDied","Data":"e5b78b23e01eb50b470839b6336e85863cc9687a745e9cd3cb7b0ae94e69dbb4"}
Sep 15 22:45:37.740399 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.740373    2578 scope.go:117] "RemoveContainer" containerID="4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e"
Sep 15 22:45:37.740706 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.740565    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"
Sep 15 22:45:37.751992 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.751969    2578 scope.go:117] "RemoveContainer" containerID="4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e"
Sep 15 22:45:37.752309 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:45:37.752287    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e\": container with ID starting with 4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e not found: ID does not exist" containerID="4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e"
Sep 15 22:45:37.752375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.752319    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e"} err="failed to get container status \"4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e\": rpc error: code = NotFound desc = could not find container \"4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e\": container with ID starting with 4ebe1c8aa5e959100361f785cba94b15db794fefea061c0686924171c37d595e not found: ID does not exist"
Sep 15 22:45:37.764953 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.764925    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"]
Sep 15 22:45:37.768158 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:37.768135    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bbfe736-69dfd845cf-97j66"]
Sep 15 22:45:38.762871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:38.762836    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e45e2df6-27fc-4f77-9881-ee93d1f23346" path="/var/lib/kubelet/pods/e45e2df6-27fc-4f77-9881-ee93d1f23346/volumes"
Sep 15 22:45:43.683022 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:43.682986    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"]
Sep 15 22:45:43.683451 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:43.683314    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy" containerID="cri-o://1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b" gracePeriod=30
Sep 15 22:45:48.944628 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:48.944547    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:45:49.079614 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.079574    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-credential-socket\") pod \"182f5bf6-5bc8-413a-beb4-de67f156251a\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") "
Sep 15 22:45:49.079809 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.079640    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182f5bf6-5bc8-413a-beb4-de67f156251a-istiod-ca-cert\") pod \"182f5bf6-5bc8-413a-beb4-de67f156251a\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") "
Sep 15 22:45:49.079809 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.079681    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-socket\") pod \"182f5bf6-5bc8-413a-beb4-de67f156251a\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") "
Sep 15 22:45:49.079809 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.079721    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-envoy\") pod \"182f5bf6-5bc8-413a-beb4-de67f156251a\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") "
Sep 15 22:45:49.079809 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.079794    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-podinfo\") pod \"182f5bf6-5bc8-413a-beb4-de67f156251a\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") "
Sep 15 22:45:49.080050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.079862    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-data\") pod \"182f5bf6-5bc8-413a-beb4-de67f156251a\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") "
Sep 15 22:45:49.080050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.079952    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-certs\") pod \"182f5bf6-5bc8-413a-beb4-de67f156251a\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") "
Sep 15 22:45:49.080050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.079982    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvpdx\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-kube-api-access-vvpdx\") pod \"182f5bf6-5bc8-413a-beb4-de67f156251a\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") "
Sep 15 22:45:49.080050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080012    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-token\") pod \"182f5bf6-5bc8-413a-beb4-de67f156251a\" (UID: \"182f5bf6-5bc8-413a-beb4-de67f156251a\") "
Sep 15 22:45:49.080286 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.079975    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "182f5bf6-5bc8-413a-beb4-de67f156251a" (UID: "182f5bf6-5bc8-413a-beb4-de67f156251a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:45:49.080286 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080180    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-data" (OuterVolumeSpecName: "istio-data") pod "182f5bf6-5bc8-413a-beb4-de67f156251a" (UID: "182f5bf6-5bc8-413a-beb4-de67f156251a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:45:49.080286 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080156    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "182f5bf6-5bc8-413a-beb4-de67f156251a" (UID: "182f5bf6-5bc8-413a-beb4-de67f156251a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:45:49.080286 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080256    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "182f5bf6-5bc8-413a-beb4-de67f156251a" (UID: "182f5bf6-5bc8-413a-beb4-de67f156251a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:45:49.080475 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080284    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/182f5bf6-5bc8-413a-beb4-de67f156251a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "182f5bf6-5bc8-413a-beb4-de67f156251a" (UID: "182f5bf6-5bc8-413a-beb4-de67f156251a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:45:49.080529 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080507    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182f5bf6-5bc8-413a-beb4-de67f156251a-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:49.080529 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080527    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:49.080652 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080537    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:49.080652 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080548    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:49.080652 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.080560    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:49.082358 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.082323    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "182f5bf6-5bc8-413a-beb4-de67f156251a" (UID: "182f5bf6-5bc8-413a-beb4-de67f156251a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:45:49.082479 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.082363    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "182f5bf6-5bc8-413a-beb4-de67f156251a" (UID: "182f5bf6-5bc8-413a-beb4-de67f156251a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:45:49.082479 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.082440    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-token" (OuterVolumeSpecName: "istio-token") pod "182f5bf6-5bc8-413a-beb4-de67f156251a" (UID: "182f5bf6-5bc8-413a-beb4-de67f156251a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:45:49.082559 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.082471    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-kube-api-access-vvpdx" (OuterVolumeSpecName: "kube-api-access-vvpdx") pod "182f5bf6-5bc8-413a-beb4-de67f156251a" (UID: "182f5bf6-5bc8-413a-beb4-de67f156251a"). InnerVolumeSpecName "kube-api-access-vvpdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:45:49.181341 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.181301    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvpdx\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-kube-api-access-vvpdx\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:49.181341 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.181335    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:49.181341 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.181345    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:49.181341 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.181353    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182f5bf6-5bc8-413a-beb4-de67f156251a-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:45:49.800693 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.800651    2578 generic.go:356] "Generic (PLEG): container finished" podID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerID="1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b" exitCode=0
Sep 15 22:45:49.800880 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.800714    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"
Sep 15 22:45:49.800880 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.800735    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" event={"ID":"182f5bf6-5bc8-413a-beb4-de67f156251a","Type":"ContainerDied","Data":"1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b"}
Sep 15 22:45:49.800880 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.800792    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn" event={"ID":"182f5bf6-5bc8-413a-beb4-de67f156251a","Type":"ContainerDied","Data":"df70729726b22a41685ea3bac256259ea1f84440bc62854581c94be8d0e36967"}
Sep 15 22:45:49.800880 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.800815    2578 scope.go:117] "RemoveContainer" containerID="1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b"
Sep 15 22:45:49.810451 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.810434    2578 scope.go:117] "RemoveContainer" containerID="e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399"
Sep 15 22:45:49.822337 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.822309    2578 scope.go:117] "RemoveContainer" containerID="1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b"
Sep 15 22:45:49.822674 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:45:49.822645    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b\": container with ID starting with 1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b not found: ID does not exist" containerID="1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b"
Sep 15 22:45:49.822740 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.822687    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b"} err="failed to get container status \"1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b\": rpc error: code = NotFound desc = could not find container \"1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b\": container with ID starting with 1ef3577d2497543ff42c8c8efa43352e44d8b24b103619903573d9e3bdb87d3b not found: ID does not exist"
Sep 15 22:45:49.822740 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.822713    2578 scope.go:117] "RemoveContainer" containerID="e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399"
Sep 15 22:45:49.822975 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:45:49.822957    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399\": container with ID starting with e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399 not found: ID does not exist" containerID="e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399"
Sep 15 22:45:49.823028 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.822980    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399"} err="failed to get container status \"e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399\": rpc error: code = NotFound desc = could not find container \"e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399\": container with ID starting with e615689a405247853d8f54293390bacc55435550ecff34c2bf5ff47f10c20399 not found: ID does not exist"
Sep 15 22:45:49.824040 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.824021    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"]
Sep 15 22:45:49.826951 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:49.826928    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2bbfe736-openshift-default-76d849cc48-gxkhn"]
Sep 15 22:45:50.763258 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:50.763224    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" path="/var/lib/kubelet/pods/182f5bf6-5bc8-413a-beb4-de67f156251a/volumes"
Sep 15 22:45:57.625874 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.625843    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"]
Sep 15 22:45:57.626268 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626251    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f480852-191c-4f26-af93-622acfa750fe" containerName="cleanup"
Sep 15 22:45:57.626268 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626264    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f480852-191c-4f26-af93-622acfa750fe" containerName="cleanup"
Sep 15 22:45:57.626353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626272    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy"
Sep 15 22:45:57.626353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626277    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy"
Sep 15 22:45:57.626353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626291    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45e2df6-27fc-4f77-9881-ee93d1f23346" containerName="maas-api"
Sep 15 22:45:57.626353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626296    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45e2df6-27fc-4f77-9881-ee93d1f23346" containerName="maas-api"
Sep 15 22:45:57.626353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626304    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy"
Sep 15 22:45:57.626353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626309    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy"
Sep 15 22:45:57.626547 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626385    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy"
Sep 15 22:45:57.626547 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626395    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f480852-191c-4f26-af93-622acfa750fe" containerName="cleanup"
Sep 15 22:45:57.626547 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626404    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="182f5bf6-5bc8-413a-beb4-de67f156251a" containerName="istio-proxy"
Sep 15 22:45:57.626547 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.626411    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45e2df6-27fc-4f77-9881-ee93d1f23346" containerName="maas-api"
Sep 15 22:45:57.635348 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.635322    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.638424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.638391    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-f2241f-openshift-default-dockercfg-s576k\""
Sep 15 22:45:57.643945 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.643919    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"]
Sep 15 22:45:57.763399 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.763366    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-istio-token\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.763399 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.763403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-data\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.763641 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.763425    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-credential-socket\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.763641 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.763493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/058dbafa-8205-4b61-b96c-87d6a729374d-istiod-ca-cert\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.763641 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.763545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-certs\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.763641 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.763561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/058dbafa-8205-4b61-b96c-87d6a729374d-istio-podinfo\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.763641 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.763586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-envoy\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.763641 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.763614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp2l5\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-kube-api-access-dp2l5\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.763641 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.763634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-socket\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865052 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-credential-socket\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/058dbafa-8205-4b61-b96c-87d6a729374d-istiod-ca-cert\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-certs\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/058dbafa-8205-4b61-b96c-87d6a729374d-istio-podinfo\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865306 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-envoy\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp2l5\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-kube-api-access-dp2l5\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865339    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-socket\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-istio-token\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-data\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865747 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-certs\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865811 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865755    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-socket\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865922 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-credential-socket\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.865922 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865914    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-data\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.866085 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.865917    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/058dbafa-8205-4b61-b96c-87d6a729374d-istiod-ca-cert\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.867677 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.867658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-envoy\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.867677 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.867672    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/058dbafa-8205-4b61-b96c-87d6a729374d-istio-podinfo\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.873859 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.873825    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-istio-token\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.874061 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.874038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp2l5\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-kube-api-access-dp2l5\") pod \"e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") " pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:57.947700 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:57.947656    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:58.225476 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:58.225449    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"]
Sep 15 22:45:58.228264 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:45:58.228221    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod058dbafa_8205_4b61_b96c_87d6a729374d.slice/crio-d38b2b00914a9819a344f548dc03e61f932de61322e61213db6d3d641a6eda80 WatchSource:0}: Error finding container d38b2b00914a9819a344f548dc03e61f932de61322e61213db6d3d641a6eda80: Status 404 returned error can't find the container with id d38b2b00914a9819a344f548dc03e61f932de61322e61213db6d3d641a6eda80
Sep 15 22:45:58.230581 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:58.230540    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:45:58.230677 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:58.230613    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:45:58.230677 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:58.230647    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:45:58.846608 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:58.846573    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" event={"ID":"058dbafa-8205-4b61-b96c-87d6a729374d","Type":"ContainerStarted","Data":"50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9"}
Sep 15 22:45:58.847071 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:58.846633    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" event={"ID":"058dbafa-8205-4b61-b96c-87d6a729374d","Type":"ContainerStarted","Data":"d38b2b00914a9819a344f548dc03e61f932de61322e61213db6d3d641a6eda80"}
Sep 15 22:45:58.868888 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:58.868823    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" podStartSLOduration=1.868798654 podStartE2EDuration="1.868798654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:45:58.867366218 +0000 UTC m=+1098.620990350" watchObservedRunningTime="2026-09-15 22:45:58.868798654 +0000 UTC m=+1098.622422782"
Sep 15 22:45:58.947856 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:58.947823    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:45:59.953466 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:45:59.953438    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:46:00.857936 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:00.857904    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:46:00.859132 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:00.859107    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:46:21.126622 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.126586    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"]
Sep 15 22:46:21.130554 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.130523    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.133208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.133182    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-4f70a9-openshift-default-dockercfg-7sjkb\""
Sep 15 22:46:21.141878 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.141850    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"]
Sep 15 22:46:21.191360 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.191323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istiod-ca-cert\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.191360 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.191360    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q62lv\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-kube-api-access-q62lv\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.191581 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.191441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-credential-socket\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.191581 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.191461    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-envoy\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.191581 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.191484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-certs\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.191581 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.191521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-socket\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.191708 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.191576    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-podinfo\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.191708 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.191665    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-token\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.191778 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.191711    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-data\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.292434 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-token\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.292434 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-data\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.292733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istiod-ca-cert\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.292733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q62lv\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-kube-api-access-q62lv\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.292733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292569    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-credential-socket\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.292733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292597    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-envoy\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.292733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-certs\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.292733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-socket\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.292733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-podinfo\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.293142 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.292937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-data\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.293142 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.293011    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-credential-socket\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.293321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.293299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-certs\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.293375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.293348    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-socket\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.293417 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.293350    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istiod-ca-cert\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.294906 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.294883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-envoy\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.295163 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.295146    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-podinfo\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.300814 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.300775    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-token\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.300919 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.300818    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q62lv\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-kube-api-access-q62lv\") pod \"e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") " pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.444379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.444339    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:21.724025 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:46:21.723983    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b84e575_ed0f_42a3_a8b1_e4122a09997e.slice/crio-a3ed126361e105f81247d32e0c06595d351d361a5ab664900d23d27d15848876 WatchSource:0}: Error finding container a3ed126361e105f81247d32e0c06595d351d361a5ab664900d23d27d15848876: Status 404 returned error can't find the container with id a3ed126361e105f81247d32e0c06595d351d361a5ab664900d23d27d15848876
Sep 15 22:46:21.724178 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.724065    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"]
Sep 15 22:46:21.726837 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.726801    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:21.726949 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.726870    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:21.726949 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.726896    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:21.961480 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.961437    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" event={"ID":"1b84e575-ed0f-42a3-a8b1-e4122a09997e","Type":"ContainerStarted","Data":"58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403"}
Sep 15 22:46:21.961669 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.961511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" event={"ID":"1b84e575-ed0f-42a3-a8b1-e4122a09997e","Type":"ContainerStarted","Data":"a3ed126361e105f81247d32e0c06595d351d361a5ab664900d23d27d15848876"}
Sep 15 22:46:21.986650 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:21.986532    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" podStartSLOduration=0.98651307 podStartE2EDuration="986.51307ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:46:21.984874417 +0000 UTC m=+1121.738498558" watchObservedRunningTime="2026-09-15 22:46:21.98651307 +0000 UTC m=+1121.740137197"
Sep 15 22:46:22.445301 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:22.445267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:22.450790 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:22.450761    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:22.966013 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:22.965983    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:22.966956 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:22.966931    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:30.508634 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.508599    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"]
Sep 15 22:46:30.511293 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.511263    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.514050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.514023    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-1cc224-openshift-default-dockercfg-xfprs\""
Sep 15 22:46:30.525973 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.525944    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"]
Sep 15 22:46:30.601030 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.600948    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-certs\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.601030 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.600984    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-envoy\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.601030 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.601000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-podinfo\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.601288 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.601072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hljt2\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-kube-api-access-hljt2\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.601288 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.601190    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-socket\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.601288 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.601207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-token\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.601288 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.601227    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-credential-socket\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.601288 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.601252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istiod-ca-cert\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.601445 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.601289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-data\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.702994 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.702952    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-socket\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.702994 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-token\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703302 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-credential-socket\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703302 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istiod-ca-cert\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703302 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-data\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703456 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-certs\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703456 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-envoy\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703564 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-podinfo\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703564 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703470    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-socket\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703564 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-credential-socket\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703564 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hljt2\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-kube-api-access-hljt2\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703789 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703611    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-data\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703850 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-certs\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.703904 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.703886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istiod-ca-cert\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.707115 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.706783    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-envoy\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.707115 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.707082    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-podinfo\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.714745 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.714709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-token\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.714914 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.714793    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hljt2\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-kube-api-access-hljt2\") pod \"e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") " pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.823071 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.823031    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:30.860069 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.860029    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 15 22:46:30.860255 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:30.860117    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 15 22:46:31.014392 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.014299    2578 generic.go:356] "Generic (PLEG): container finished" podID="058dbafa-8205-4b61-b96c-87d6a729374d" containerID="50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9" exitCode=0
Sep 15 22:46:31.014564 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.014377    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" event={"ID":"058dbafa-8205-4b61-b96c-87d6a729374d","Type":"ContainerDied","Data":"50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9"}
Sep 15 22:46:31.014909 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.014884    2578 scope.go:117] "RemoveContainer" containerID="50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9"
Sep 15 22:46:31.016138 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.016079    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:31.016264 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.016180    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:31.016264 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.016222    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:31.101543 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.101503    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"]
Sep 15 22:46:31.103283 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:46:31.103245    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff3161e_65c8_42a1_88e0_a3ba8e822bb2.slice/crio-f2e3fb2f3ded1172a3a8df333c71711e23d8d247cccf087f52dc9f0bda0693b4 WatchSource:0}: Error finding container f2e3fb2f3ded1172a3a8df333c71711e23d8d247cccf087f52dc9f0bda0693b4: Status 404 returned error can't find the container with id f2e3fb2f3ded1172a3a8df333c71711e23d8d247cccf087f52dc9f0bda0693b4
Sep 15 22:46:31.106000 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.105964    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:31.106167 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.106069    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:31.106167 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:31.106137    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:32.023345 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:32.023309    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" event={"ID":"058dbafa-8205-4b61-b96c-87d6a729374d","Type":"ContainerStarted","Data":"7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83"}
Sep 15 22:46:32.024860 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:32.024830    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj" event={"ID":"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2","Type":"ContainerStarted","Data":"1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1"}
Sep 15 22:46:32.024987 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:32.024881    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj" event={"ID":"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2","Type":"ContainerStarted","Data":"f2e3fb2f3ded1172a3a8df333c71711e23d8d247cccf087f52dc9f0bda0693b4"}
Sep 15 22:46:32.067541 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:32.067485    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj" podStartSLOduration=2.067466166 podStartE2EDuration="2.067466166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:46:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:46:32.064657059 +0000 UTC m=+1131.818281168" watchObservedRunningTime="2026-09-15 22:46:32.067466166 +0000 UTC m=+1131.821090294"
Sep 15 22:46:32.823775 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:32.823739    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:32.829170 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:32.829141    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:32.948475 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:32.948438    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:46:32.948831 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:32.948801    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 15 22:46:32.948970 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:32.948851    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 15 22:46:33.030572 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:33.030543    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:33.031271 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:33.031253    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:46:33.948903 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:33.948859    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 15 22:46:33.949088 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:33.948932    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 15 22:46:34.948914 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:34.948871    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 15 22:46:34.949349 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:34.948943    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 15 22:46:35.948974 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:35.948930    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 15 22:46:35.949434 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:35.948992    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 15 22:46:36.948861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:36.948823    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 15 22:46:36.949040 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:36.948894    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 15 22:46:37.952336 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:37.952295    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:46:37.952751 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:37.952625    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:46:37.953520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:37.953501    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:46:50.125420 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:50.125383    2578 generic.go:356] "Generic (PLEG): container finished" podID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerID="58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403" exitCode=0
Sep 15 22:46:50.125859 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:50.125452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" event={"ID":"1b84e575-ed0f-42a3-a8b1-e4122a09997e","Type":"ContainerDied","Data":"58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403"}
Sep 15 22:46:50.125922 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:50.125908    2578 scope.go:117] "RemoveContainer" containerID="58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403"
Sep 15 22:46:50.127139 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:50.127082    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:50.127200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:50.127181    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:50.127239 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:50.127212    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:46:51.138358 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:51.138307    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" event={"ID":"1b84e575-ed0f-42a3-a8b1-e4122a09997e","Type":"ContainerStarted","Data":"69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d"}
Sep 15 22:46:51.444625 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:51.444593    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:51.445832 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:51.445808    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 22:46:51.445912 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:51.445876    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 22:46:52.445192 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:52.445147    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 22:46:52.445672 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:52.445210    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 22:46:52.966283 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:52.966242    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:53.445193 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:53.445152    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 22:46:53.445373 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:53.445209    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 22:46:54.445312 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:54.445266    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 22:46:54.445511 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:54.445328    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 22:46:55.445778 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:55.445741    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 22:46:55.446181 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:55.445798    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 22:46:56.444895 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:56.444858    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 22:46:56.445087 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:56.444912    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 22:46:57.449351 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:57.449318    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:46:57.450431 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:46:57.450411    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:47:22.750475 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.750394    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"]
Sep 15 22:47:22.753952 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.753935    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.756933 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.756910    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-1cc224\"/\"kube-root-ca.crt\""
Sep 15 22:47:22.758085 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.758065    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-1cc224\"/\"default-dockercfg-bdccn\""
Sep 15 22:47:22.758230 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.758158    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-1cc224\"/\"noten-model-1cc224-kserve-self-signed-certs\""
Sep 15 22:47:22.758230 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.758209    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-1cc224\"/\"openshift-service-ca.crt\""
Sep 15 22:47:22.765235 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.765209    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"]
Sep 15 22:47:22.852636 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.852606    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tmp-dir\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.852826 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.852667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-home\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.852826 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.852685    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-dshm\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.852826 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.852785    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-model-cache\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.852826 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.852810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz458\" (UniqueName: \"kubernetes.io/projected/fb587b60-88aa-4df8-bbb6-5608f0b63a66-kube-api-access-tz458\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.853010 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.852833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tls-certs\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.953817 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.953779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-model-cache\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.953996 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.953820    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz458\" (UniqueName: \"kubernetes.io/projected/fb587b60-88aa-4df8-bbb6-5608f0b63a66-kube-api-access-tz458\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.953996 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.953938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tls-certs\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.954154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.954009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tmp-dir\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.954154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.954072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-home\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.954154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.954120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-dshm\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.954325 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.954302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-model-cache\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.954383 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.954330    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tmp-dir\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.954440 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.954398    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-home\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.956341 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.956316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-dshm\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.956518 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.956499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tls-certs\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:22.962765 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:22.962743    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz458\" (UniqueName: \"kubernetes.io/projected/fb587b60-88aa-4df8-bbb6-5608f0b63a66-kube-api-access-tz458\") pod \"noten-model-1cc224-kserve-7697db8b59-9vnfg\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") " pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:23.064881 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:23.064789    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:23.338158 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:23.338062    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"]
Sep 15 22:47:23.341686 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:47:23.341642    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb587b60_88aa_4df8_bbb6_5608f0b63a66.slice/crio-824ab773be5917a9a772825be49e25b313d67315cad66f8f763d31dc50238b61 WatchSource:0}: Error finding container 824ab773be5917a9a772825be49e25b313d67315cad66f8f763d31dc50238b61: Status 404 returned error can't find the container with id 824ab773be5917a9a772825be49e25b313d67315cad66f8f763d31dc50238b61
Sep 15 22:47:24.316633 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:24.316597    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg" event={"ID":"fb587b60-88aa-4df8-bbb6-5608f0b63a66","Type":"ContainerStarted","Data":"6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63"}
Sep 15 22:47:24.316633 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:24.316639    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg" event={"ID":"fb587b60-88aa-4df8-bbb6-5608f0b63a66","Type":"ContainerStarted","Data":"824ab773be5917a9a772825be49e25b313d67315cad66f8f763d31dc50238b61"}
Sep 15 22:47:24.334519 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:24.334464    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg" podStartSLOduration=2.334447557 podStartE2EDuration="2.334447557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:47:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:47:24.333343002 +0000 UTC m=+1184.086967131" watchObservedRunningTime="2026-09-15 22:47:24.334447557 +0000 UTC m=+1184.088071685"
Sep 15 22:47:33.065481 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:33.065438    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:33.065948 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:33.065506    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:33.077812 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:33.077785    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:33.380550 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:33.380465    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:47:55.528856 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:55.528819    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"]
Sep 15 22:47:55.529439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:47:55.529044    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj" podUID="2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" containerName="istio-proxy" containerID="cri-o://1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1" gracePeriod=30
Sep 15 22:48:00.777541 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.777515    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:48:00.840541 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.840444    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"]
Sep 15 22:48:00.850572 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850536    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hljt2\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-kube-api-access-hljt2\") pod \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") "
Sep 15 22:48:00.850741 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850592    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-data\") pod \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") "
Sep 15 22:48:00.850741 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850618    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-envoy\") pod \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") "
Sep 15 22:48:00.850741 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850656    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-socket\") pod \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") "
Sep 15 22:48:00.850741 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850691    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-credential-socket\") pod \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") "
Sep 15 22:48:00.850741 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850722    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istiod-ca-cert\") pod \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") "
Sep 15 22:48:00.851016 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850763    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-token\") pod \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") "
Sep 15 22:48:00.851016 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850843    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-podinfo\") pod \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") "
Sep 15 22:48:00.851016 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850903    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-certs\") pod \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\" (UID: \"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2\") "
Sep 15 22:48:00.851016 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.850936    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" (UID: "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:00.851537 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.851267    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" (UID: "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:48:00.851537 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.851343    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:00.851537 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.851364    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:00.851537 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.851450    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" (UID: "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:00.851537 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.851505    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" (UID: "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:00.851537 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.851522    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" (UID: "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:00.853257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.853223    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" (UID: "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:48:00.853385 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.853312    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" (UID: "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:00.853550 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.853529    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" (UID: "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:48:00.853733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.853717    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-kube-api-access-hljt2" (OuterVolumeSpecName: "kube-api-access-hljt2") pod "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" (UID: "2ff3161e-65c8-42a1-88e0-a3ba8e822bb2"). InnerVolumeSpecName "kube-api-access-hljt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:48:00.952439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.952404    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:00.952439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.952432    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hljt2\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-kube-api-access-hljt2\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:00.952439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.952443    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:00.952704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.952451    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:00.952704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.952460    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:00.952704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.952468    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:00.952704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:00.952476    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:01.510204 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:01.510169    2578 generic.go:356] "Generic (PLEG): container finished" podID="2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" containerID="1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1" exitCode=0
Sep 15 22:48:01.510520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:01.510235    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"
Sep 15 22:48:01.510520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:01.510258    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj" event={"ID":"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2","Type":"ContainerDied","Data":"1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1"}
Sep 15 22:48:01.510520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:01.510352    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj" event={"ID":"2ff3161e-65c8-42a1-88e0-a3ba8e822bb2","Type":"ContainerDied","Data":"f2e3fb2f3ded1172a3a8df333c71711e23d8d247cccf087f52dc9f0bda0693b4"}
Sep 15 22:48:01.510520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:01.510389    2578 scope.go:117] "RemoveContainer" containerID="1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1"
Sep 15 22:48:01.520256 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:01.520234    2578 scope.go:117] "RemoveContainer" containerID="1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1"
Sep 15 22:48:01.520558 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:48:01.520537    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1\": container with ID starting with 1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1 not found: ID does not exist" containerID="1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1"
Sep 15 22:48:01.520627 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:01.520568    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1"} err="failed to get container status \"1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1\": rpc error: code = NotFound desc = could not find container \"1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1\": container with ID starting with 1b4f23d1e0682a63a630ad3d28c00c00be28bb1b128026976c6d54506b198fd1 not found: ID does not exist"
Sep 15 22:48:01.532822 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:01.532781    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"]
Sep 15 22:48:01.538052 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:01.538024    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-1cc224-openshift-default-644d7cf99d-f9trj"]
Sep 15 22:48:02.763087 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:02.763048    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" path="/var/lib/kubelet/pods/2ff3161e-65c8-42a1-88e0-a3ba8e822bb2/volumes"
Sep 15 22:48:15.366213 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:15.366180    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"]
Sep 15 22:48:15.861601 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:15.861534    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg" podUID="fb587b60-88aa-4df8-bbb6-5608f0b63a66" containerName="main" containerID="cri-o://6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63" gracePeriod=45
Sep 15 22:48:16.116599 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.116532    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:48:16.214207 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214167    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-dshm\") pod \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") "
Sep 15 22:48:16.214402 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214222    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-home\") pod \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") "
Sep 15 22:48:16.214402 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214283    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tls-certs\") pod \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") "
Sep 15 22:48:16.214402 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214307    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz458\" (UniqueName: \"kubernetes.io/projected/fb587b60-88aa-4df8-bbb6-5608f0b63a66-kube-api-access-tz458\") pod \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") "
Sep 15 22:48:16.214402 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214382    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tmp-dir\") pod \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") "
Sep 15 22:48:16.214613 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214426    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-model-cache\") pod \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\" (UID: \"fb587b60-88aa-4df8-bbb6-5608f0b63a66\") "
Sep 15 22:48:16.214613 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214563    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-home" (OuterVolumeSpecName: "home") pod "fb587b60-88aa-4df8-bbb6-5608f0b63a66" (UID: "fb587b60-88aa-4df8-bbb6-5608f0b63a66"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:16.214715 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214668    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb587b60-88aa-4df8-bbb6-5608f0b63a66" (UID: "fb587b60-88aa-4df8-bbb6-5608f0b63a66"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:16.214796 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214765    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb587b60-88aa-4df8-bbb6-5608f0b63a66" (UID: "fb587b60-88aa-4df8-bbb6-5608f0b63a66"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:16.214917 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214902    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:16.214962 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214922    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:16.214962 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.214937    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:16.216691 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.216670    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb587b60-88aa-4df8-bbb6-5608f0b63a66-kube-api-access-tz458" (OuterVolumeSpecName: "kube-api-access-tz458") pod "fb587b60-88aa-4df8-bbb6-5608f0b63a66" (UID: "fb587b60-88aa-4df8-bbb6-5608f0b63a66"). InnerVolumeSpecName "kube-api-access-tz458". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:48:16.216758 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.216738    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb587b60-88aa-4df8-bbb6-5608f0b63a66" (UID: "fb587b60-88aa-4df8-bbb6-5608f0b63a66"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:48:16.216806 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.216759    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-dshm" (OuterVolumeSpecName: "dshm") pod "fb587b60-88aa-4df8-bbb6-5608f0b63a66" (UID: "fb587b60-88aa-4df8-bbb6-5608f0b63a66"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:16.316557 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.316520    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb587b60-88aa-4df8-bbb6-5608f0b63a66-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:16.316557 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.316553    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb587b60-88aa-4df8-bbb6-5608f0b63a66-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:16.316557 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.316564    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz458\" (UniqueName: \"kubernetes.io/projected/fb587b60-88aa-4df8-bbb6-5608f0b63a66-kube-api-access-tz458\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:16.588389 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.588354    2578 generic.go:356] "Generic (PLEG): container finished" podID="fb587b60-88aa-4df8-bbb6-5608f0b63a66" containerID="6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63" exitCode=0
Sep 15 22:48:16.588818 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.588395    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"
Sep 15 22:48:16.588818 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.588439    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg" event={"ID":"fb587b60-88aa-4df8-bbb6-5608f0b63a66","Type":"ContainerDied","Data":"6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63"}
Sep 15 22:48:16.588818 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.588481    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg" event={"ID":"fb587b60-88aa-4df8-bbb6-5608f0b63a66","Type":"ContainerDied","Data":"824ab773be5917a9a772825be49e25b313d67315cad66f8f763d31dc50238b61"}
Sep 15 22:48:16.588818 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.588498    2578 scope.go:117] "RemoveContainer" containerID="6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63"
Sep 15 22:48:16.597701 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.597685    2578 scope.go:117] "RemoveContainer" containerID="6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63"
Sep 15 22:48:16.598002 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:48:16.597982    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63\": container with ID starting with 6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63 not found: ID does not exist" containerID="6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63"
Sep 15 22:48:16.598068 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.598015    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63"} err="failed to get container status \"6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63\": rpc error: code = NotFound desc = could not find container \"6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63\": container with ID starting with 6fb7c6dda9e2abfacd64c1639f7fcd3179c65ffb506a541b07e5099459564c63 not found: ID does not exist"
Sep 15 22:48:16.610488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.610451    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"]
Sep 15 22:48:16.614133 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.614106    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-1cc224/noten-model-1cc224-kserve-7697db8b59-9vnfg"]
Sep 15 22:48:16.762882 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:16.762851    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb587b60-88aa-4df8-bbb6-5608f0b63a66" path="/var/lib/kubelet/pods/fb587b60-88aa-4df8-bbb6-5608f0b63a66/volumes"
Sep 15 22:48:30.386088 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.386026    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm" podUID="c50aa469-ded4-4043-b58e-450ab73f2ac4" containerName="main" containerID="cri-o://31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90" gracePeriod=45
Sep 15 22:48:30.643206 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.643140    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:48:30.662130 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.662028    2578 generic.go:356] "Generic (PLEG): container finished" podID="c50aa469-ded4-4043-b58e-450ab73f2ac4" containerID="31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90" exitCode=0
Sep 15 22:48:30.662130 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.662112    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"
Sep 15 22:48:30.662130 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.662131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm" event={"ID":"c50aa469-ded4-4043-b58e-450ab73f2ac4","Type":"ContainerDied","Data":"31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90"}
Sep 15 22:48:30.662435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.662168    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm" event={"ID":"c50aa469-ded4-4043-b58e-450ab73f2ac4","Type":"ContainerDied","Data":"afbaa160c37a557985a7a4d547df16973d7f0863b8aa074d3ce3b243090b0077"}
Sep 15 22:48:30.662435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.662189    2578 scope.go:117] "RemoveContainer" containerID="31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90"
Sep 15 22:48:30.671702 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.671671    2578 scope.go:117] "RemoveContainer" containerID="31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90"
Sep 15 22:48:30.672073 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:48:30.672051    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90\": container with ID starting with 31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90 not found: ID does not exist" containerID="31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90"
Sep 15 22:48:30.672146 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.672084    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90"} err="failed to get container status \"31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90\": rpc error: code = NotFound desc = could not find container \"31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90\": container with ID starting with 31e531bc2b13470f0ff3588f8c3d558673bf825e923666178afb0bdbc1cd9e90 not found: ID does not exist"
Sep 15 22:48:30.754493 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.754454    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c50aa469-ded4-4043-b58e-450ab73f2ac4-tls-certs\") pod \"c50aa469-ded4-4043-b58e-450ab73f2ac4\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") "
Sep 15 22:48:30.754683 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.754505    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-988th\" (UniqueName: \"kubernetes.io/projected/c50aa469-ded4-4043-b58e-450ab73f2ac4-kube-api-access-988th\") pod \"c50aa469-ded4-4043-b58e-450ab73f2ac4\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") "
Sep 15 22:48:30.754683 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.754540    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-model-cache\") pod \"c50aa469-ded4-4043-b58e-450ab73f2ac4\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") "
Sep 15 22:48:30.754683 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.754566    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-tmp-dir\") pod \"c50aa469-ded4-4043-b58e-450ab73f2ac4\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") "
Sep 15 22:48:30.754683 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.754652    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-home\") pod \"c50aa469-ded4-4043-b58e-450ab73f2ac4\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") "
Sep 15 22:48:30.754905 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.754726    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-dshm\") pod \"c50aa469-ded4-4043-b58e-450ab73f2ac4\" (UID: \"c50aa469-ded4-4043-b58e-450ab73f2ac4\") "
Sep 15 22:48:30.754905 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.754854    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-model-cache" (OuterVolumeSpecName: "model-cache") pod "c50aa469-ded4-4043-b58e-450ab73f2ac4" (UID: "c50aa469-ded4-4043-b58e-450ab73f2ac4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:30.755000 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.754893    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c50aa469-ded4-4043-b58e-450ab73f2ac4" (UID: "c50aa469-ded4-4043-b58e-450ab73f2ac4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:30.755000 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.754957    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-home" (OuterVolumeSpecName: "home") pod "c50aa469-ded4-4043-b58e-450ab73f2ac4" (UID: "c50aa469-ded4-4043-b58e-450ab73f2ac4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:30.755157 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.755137    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:30.755157 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.755155    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:30.755285 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.755163    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:30.756849 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.756813    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-dshm" (OuterVolumeSpecName: "dshm") pod "c50aa469-ded4-4043-b58e-450ab73f2ac4" (UID: "c50aa469-ded4-4043-b58e-450ab73f2ac4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:48:30.756976 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.756874    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c50aa469-ded4-4043-b58e-450ab73f2ac4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c50aa469-ded4-4043-b58e-450ab73f2ac4" (UID: "c50aa469-ded4-4043-b58e-450ab73f2ac4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:48:30.756976 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.756919    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c50aa469-ded4-4043-b58e-450ab73f2ac4-kube-api-access-988th" (OuterVolumeSpecName: "kube-api-access-988th") pod "c50aa469-ded4-4043-b58e-450ab73f2ac4" (UID: "c50aa469-ded4-4043-b58e-450ab73f2ac4"). InnerVolumeSpecName "kube-api-access-988th". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:48:30.856136 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.856074    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c50aa469-ded4-4043-b58e-450ab73f2ac4-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:30.856136 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.856133    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c50aa469-ded4-4043-b58e-450ab73f2ac4-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:30.856136 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.856146    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-988th\" (UniqueName: \"kubernetes.io/projected/c50aa469-ded4-4043-b58e-450ab73f2ac4-kube-api-access-988th\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:48:30.978776 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.978742    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"]
Sep 15 22:48:30.981941 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:30.981914    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-fffdae/test-model-fffdae-kserve-86499d5bcc-r8pqm"]
Sep 15 22:48:32.763157 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:32.763044    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c50aa469-ded4-4043-b58e-450ab73f2ac4" path="/var/lib/kubelet/pods/c50aa469-ded4-4043-b58e-450ab73f2ac4/volumes"
Sep 15 22:48:34.571429 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.571388    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"]
Sep 15 22:48:34.571953 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.571932    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb587b60-88aa-4df8-bbb6-5608f0b63a66" containerName="main"
Sep 15 22:48:34.572042 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.571957    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb587b60-88aa-4df8-bbb6-5608f0b63a66" containerName="main"
Sep 15 22:48:34.572042 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.571985    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" containerName="istio-proxy"
Sep 15 22:48:34.572042 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.571995    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" containerName="istio-proxy"
Sep 15 22:48:34.572042 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.572011    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c50aa469-ded4-4043-b58e-450ab73f2ac4" containerName="main"
Sep 15 22:48:34.572042 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.572020    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c50aa469-ded4-4043-b58e-450ab73f2ac4" containerName="main"
Sep 15 22:48:34.572297 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.572144    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ff3161e-65c8-42a1-88e0-a3ba8e822bb2" containerName="istio-proxy"
Sep 15 22:48:34.572297 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.572159    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c50aa469-ded4-4043-b58e-450ab73f2ac4" containerName="main"
Sep 15 22:48:34.572297 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.572168    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb587b60-88aa-4df8-bbb6-5608f0b63a66" containerName="main"
Sep 15 22:48:34.576419 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.576398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.580784 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.580545    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6cdcda22\"/\"openshift-service-ca.crt\""
Sep 15 22:48:34.580784 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.580558    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6cdcda22\"/\"default-dockercfg-p46rf\""
Sep 15 22:48:34.580784 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.580565    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6cdcda22\"/\"e2e-lifecycle-model-6cdcda22-kserve-self-signed-certs\""
Sep 15 22:48:34.580784 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.580674    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6cdcda22\"/\"kube-root-ca.crt\""
Sep 15 22:48:34.584340 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.584001    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"]
Sep 15 22:48:34.696440 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.696410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-tmp-dir\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.696440 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.696443    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89c7d92b-1c77-405c-b890-7eaf0474ed67-tls-certs\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.696704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.696532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-dshm\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.696704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.696589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-home\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.696704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.696647    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcfnx\" (UniqueName: \"kubernetes.io/projected/89c7d92b-1c77-405c-b890-7eaf0474ed67-kube-api-access-rcfnx\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.696704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.696680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-model-cache\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.798137 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.798080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-dshm\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.798343 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.798151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-home\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.798343 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.798186    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfnx\" (UniqueName: \"kubernetes.io/projected/89c7d92b-1c77-405c-b890-7eaf0474ed67-kube-api-access-rcfnx\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.798343 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.798298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-model-cache\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.798520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.798394    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-tmp-dir\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.798520 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.798421    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89c7d92b-1c77-405c-b890-7eaf0474ed67-tls-certs\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.798627 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.798549    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-home\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.798766 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.798734    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-tmp-dir\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.798886 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.798742    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-model-cache\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.800430 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.800403    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-dshm\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.800734 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.800718    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89c7d92b-1c77-405c-b890-7eaf0474ed67-tls-certs\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.813320 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.813294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcfnx\" (UniqueName: \"kubernetes.io/projected/89c7d92b-1c77-405c-b890-7eaf0474ed67-kube-api-access-rcfnx\") pod \"e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") " pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:34.888085 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:34.887996    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:35.158559 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:35.158469    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"]
Sep 15 22:48:35.161597 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:48:35.161568    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89c7d92b_1c77_405c_b890_7eaf0474ed67.slice/crio-60d6e34b568879cd060c615efd96029b6a78308db278219955826da901cf30d8 WatchSource:0}: Error finding container 60d6e34b568879cd060c615efd96029b6a78308db278219955826da901cf30d8: Status 404 returned error can't find the container with id 60d6e34b568879cd060c615efd96029b6a78308db278219955826da901cf30d8
Sep 15 22:48:35.691184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:35.691135    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z" event={"ID":"89c7d92b-1c77-405c-b890-7eaf0474ed67","Type":"ContainerStarted","Data":"dccdac54fcfdf1924b79a4db2b145073a01e1d1e8b1403bc41b21db1c67b9791"}
Sep 15 22:48:35.691184 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:35.691186    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z" event={"ID":"89c7d92b-1c77-405c-b890-7eaf0474ed67","Type":"ContainerStarted","Data":"60d6e34b568879cd060c615efd96029b6a78308db278219955826da901cf30d8"}
Sep 15 22:48:35.708467 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:35.708402    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z" podStartSLOduration=1.708380419 podStartE2EDuration="1.708380419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:48:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:48:35.707519848 +0000 UTC m=+1255.461143977" watchObservedRunningTime="2026-09-15 22:48:35.708380419 +0000 UTC m=+1255.462004547"
Sep 15 22:48:44.888552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:44.888508    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:44.888968 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:44.888569    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:44.901845 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:44.901816    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:48:45.759442 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:48:45.759411    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:49:07.160452 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.160420    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"]
Sep 15 22:49:07.165597 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.165572    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.168354 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.168323    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-03f258\"/\"e2e-conflict-b-51df66af-kserve-self-signed-certs\""
Sep 15 22:49:07.175452 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.175429    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"]
Sep 15 22:49:07.232576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.232537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-model-cache\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.232576 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.232587    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcmlt\" (UniqueName: \"kubernetes.io/projected/68cb1996-affd-45d3-880f-8505fa1564ea-kube-api-access-kcmlt\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.232830 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.232731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cb1996-affd-45d3-880f-8505fa1564ea-tls-certs\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.232830 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.232790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-tmp-dir\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.232931 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.232830    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-dshm\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.232931 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.232893    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-home\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.335240 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.335202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cb1996-affd-45d3-880f-8505fa1564ea-tls-certs\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.335923 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.335897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-tmp-dir\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.336205 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.336188    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-dshm\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.336439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.336422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-home\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.336620 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.336602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-model-cache\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.336758 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.336742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcmlt\" (UniqueName: \"kubernetes.io/projected/68cb1996-affd-45d3-880f-8505fa1564ea-kube-api-access-kcmlt\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.337331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.337292    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-home\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.337529 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.337502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-model-cache\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.338189 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.338162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-tmp-dir\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.341635 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.341569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cb1996-affd-45d3-880f-8505fa1564ea-tls-certs\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.342179 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.342155    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-dshm\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.347607 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.347579    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcmlt\" (UniqueName: \"kubernetes.io/projected/68cb1996-affd-45d3-880f-8505fa1564ea-kube-api-access-kcmlt\") pod \"e2e-conflict-b-51df66af-kserve-867677db86-jfngb\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") " pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.476334 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.476299    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:07.747921 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.747893    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"]
Sep 15 22:49:07.750023 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:49:07.749994    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68cb1996_affd_45d3_880f_8505fa1564ea.slice/crio-15f052be1697f25dda156e289efa3ca66e99ca310c5bd67814e4f62c9bbbb68e WatchSource:0}: Error finding container 15f052be1697f25dda156e289efa3ca66e99ca310c5bd67814e4f62c9bbbb68e: Status 404 returned error can't find the container with id 15f052be1697f25dda156e289efa3ca66e99ca310c5bd67814e4f62c9bbbb68e
Sep 15 22:49:07.859223 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.859186    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb" event={"ID":"68cb1996-affd-45d3-880f-8505fa1564ea","Type":"ContainerStarted","Data":"8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad"}
Sep 15 22:49:07.859387 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.859235    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb" event={"ID":"68cb1996-affd-45d3-880f-8505fa1564ea","Type":"ContainerStarted","Data":"15f052be1697f25dda156e289efa3ca66e99ca310c5bd67814e4f62c9bbbb68e"}
Sep 15 22:49:07.879595 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:07.879544    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb" podStartSLOduration=0.879528074 podStartE2EDuration="879.528074ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:49:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:49:07.878461343 +0000 UTC m=+1287.632085484" watchObservedRunningTime="2026-09-15 22:49:07.879528074 +0000 UTC m=+1287.633152200"
Sep 15 22:49:10.346716 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.346683    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"]
Sep 15 22:49:10.347215 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.346899    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf" podUID="04d4f5b2-d34b-4dbd-b23e-547072b855d7" containerName="maas-api" containerID="cri-o://6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1" gracePeriod=30
Sep 15 22:49:10.603251 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.603183    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:49:10.671357 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.671291    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-metrics-certs\") pod \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") "
Sep 15 22:49:10.671357 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.671342    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d4f5b2-d34b-4dbd-b23e-547072b855d7-openshift-service-ca\") pod \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") "
Sep 15 22:49:10.671975 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.671549    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d45w\" (UniqueName: \"kubernetes.io/projected/04d4f5b2-d34b-4dbd-b23e-547072b855d7-kube-api-access-9d45w\") pod \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") "
Sep 15 22:49:10.671975 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.671605    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls\") pod \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\" (UID: \"04d4f5b2-d34b-4dbd-b23e-547072b855d7\") "
Sep 15 22:49:10.671975 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.671884    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04d4f5b2-d34b-4dbd-b23e-547072b855d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04d4f5b2-d34b-4dbd-b23e-547072b855d7" (UID: "04d4f5b2-d34b-4dbd-b23e-547072b855d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:49:10.672170 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.672071    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d4f5b2-d34b-4dbd-b23e-547072b855d7-openshift-service-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:10.673845 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.673817    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "04d4f5b2-d34b-4dbd-b23e-547072b855d7" (UID: "04d4f5b2-d34b-4dbd-b23e-547072b855d7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:49:10.674088 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.674063    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d4f5b2-d34b-4dbd-b23e-547072b855d7-kube-api-access-9d45w" (OuterVolumeSpecName: "kube-api-access-9d45w") pod "04d4f5b2-d34b-4dbd-b23e-547072b855d7" (UID: "04d4f5b2-d34b-4dbd-b23e-547072b855d7"). InnerVolumeSpecName "kube-api-access-9d45w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:49:10.674224 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.674156    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04d4f5b2-d34b-4dbd-b23e-547072b855d7" (UID: "04d4f5b2-d34b-4dbd-b23e-547072b855d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:49:10.785117 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.781085    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-metrics-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:10.785117 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.781314    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d45w\" (UniqueName: \"kubernetes.io/projected/04d4f5b2-d34b-4dbd-b23e-547072b855d7-kube-api-access-9d45w\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:10.785117 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.781402    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d4f5b2-d34b-4dbd-b23e-547072b855d7-maas-api-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:10.875376 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.875279    2578 generic.go:356] "Generic (PLEG): container finished" podID="04d4f5b2-d34b-4dbd-b23e-547072b855d7" containerID="6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1" exitCode=0
Sep 15 22:49:10.875376 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.875332    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"
Sep 15 22:49:10.875376 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.875366    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf" event={"ID":"04d4f5b2-d34b-4dbd-b23e-547072b855d7","Type":"ContainerDied","Data":"6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1"}
Sep 15 22:49:10.875635 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.875423    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf" event={"ID":"04d4f5b2-d34b-4dbd-b23e-547072b855d7","Type":"ContainerDied","Data":"7e358042d87a9e37815d80ba23876b2bcd6976cbcecc1184b044858c7251930a"}
Sep 15 22:49:10.875635 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.875447    2578 scope.go:117] "RemoveContainer" containerID="6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1"
Sep 15 22:49:10.885166 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.885146    2578 scope.go:117] "RemoveContainer" containerID="6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1"
Sep 15 22:49:10.885503 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:49:10.885481    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1\": container with ID starting with 6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1 not found: ID does not exist" containerID="6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1"
Sep 15 22:49:10.885565 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.885513    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1"} err="failed to get container status \"6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1\": rpc error: code = NotFound desc = could not find container \"6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1\": container with ID starting with 6ea50fd62f2ad3d1b6f3fc76a69b37609a71c66866640583b8155798001b11a1 not found: ID does not exist"
Sep 15 22:49:10.892793 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.892767    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"]
Sep 15 22:49:10.896443 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:10.896421    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-765e6d-568854d466-bwmvf"]
Sep 15 22:49:12.762751 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:12.762715    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04d4f5b2-d34b-4dbd-b23e-547072b855d7" path="/var/lib/kubelet/pods/04d4f5b2-d34b-4dbd-b23e-547072b855d7/volumes"
Sep 15 22:49:16.982384 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:16.982353    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"]
Sep 15 22:49:16.982847 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:16.982630    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy" containerID="cri-o://d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324" gracePeriod=30
Sep 15 22:49:17.476858 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.476813    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:17.477112 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.477070    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:17.490371 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.490338    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:17.825673 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.825587    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"]
Sep 15 22:49:17.825981 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.825969    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d4f5b2-d34b-4dbd-b23e-547072b855d7" containerName="maas-api"
Sep 15 22:49:17.826031 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.825983    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d4f5b2-d34b-4dbd-b23e-547072b855d7" containerName="maas-api"
Sep 15 22:49:17.826075 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.826049    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d4f5b2-d34b-4dbd-b23e-547072b855d7" containerName="maas-api"
Sep 15 22:49:17.830409 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.830384    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:17.833135 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.833111    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-0016d4-openshift-default-dockercfg-ngv9k\""
Sep 15 22:49:17.844492 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.844456    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"]
Sep 15 22:49:17.923191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.923157    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:49:17.957906 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.957873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-credential-socket\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:17.957906 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.957909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-socket\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:17.958200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.957933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-data\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:17.958200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.958013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e12005-9215-41e3-8808-c54d4bfca288-istio-podinfo\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:17.958200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.958065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-envoy\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:17.958200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.958122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e12005-9215-41e3-8808-c54d4bfca288-istiod-ca-cert\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:17.958200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.958142    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67pt5\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-kube-api-access-67pt5\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:17.958200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.958174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-istio-token\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:17.958200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:17.958192    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-certs\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.059484    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-credential-socket\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.059543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-socket\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.059576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-data\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.059620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e12005-9215-41e3-8808-c54d4bfca288-istio-podinfo\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.059660    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-envoy\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.059704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e12005-9215-41e3-8808-c54d4bfca288-istiod-ca-cert\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.059728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67pt5\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-kube-api-access-67pt5\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.059790    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-istio-token\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.059816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-certs\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.060663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.060345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-certs\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.061655 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.060960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-credential-socket\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.061655 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.061213    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-socket\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.061655 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.061435    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-data\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.063427 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.062880    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e12005-9215-41e3-8808-c54d4bfca288-istiod-ca-cert\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.064836 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.064807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-envoy\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.071685 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.071630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67pt5\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-kube-api-access-67pt5\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.071840 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.071793    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-istio-token\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.085358 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.083610    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e12005-9215-41e3-8808-c54d4bfca288-istio-podinfo\") pod \"e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") " pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.140757 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.140719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:18.419208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.419183    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"]
Sep 15 22:49:18.421078 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:49:18.421051    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e12005_9215_41e3_8808_c54d4bfca288.slice/crio-3bef1139efc533b717aa9b4c5db30d240dca75e8af776a9b269609d9617f1422 WatchSource:0}: Error finding container 3bef1139efc533b717aa9b4c5db30d240dca75e8af776a9b269609d9617f1422: Status 404 returned error can't find the container with id 3bef1139efc533b717aa9b4c5db30d240dca75e8af776a9b269609d9617f1422
Sep 15 22:49:18.423648 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.423614    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:49:18.423755 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.423679    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:49:18.423755 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.423709    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:49:18.918160 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.918124    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" event={"ID":"09e12005-9215-41e3-8808-c54d4bfca288","Type":"ContainerStarted","Data":"32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe"}
Sep 15 22:49:18.918356 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.918182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" event={"ID":"09e12005-9215-41e3-8808-c54d4bfca288","Type":"ContainerStarted","Data":"3bef1139efc533b717aa9b4c5db30d240dca75e8af776a9b269609d9617f1422"}
Sep 15 22:49:18.940875 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:18.940819    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" podStartSLOduration=1.9408018280000001 podStartE2EDuration="1.940801828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:49:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:49:18.93943313 +0000 UTC m=+1298.693057258" watchObservedRunningTime="2026-09-15 22:49:18.940801828 +0000 UTC m=+1298.694425956"
Sep 15 22:49:19.140899 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:19.140859    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:19.146224 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:19.146198    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:19.923186 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:19.923152    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:19.924162 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:19.924140    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:20.266413 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:20.266370    2578 patch_prober.go:28] interesting pod/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 22:49:20.266797 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:20.266437    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 22:49:22.241446 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.241420    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:49:22.404955 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.404861    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-certs\") pod \"89221884-c915-4606-a1ac-3d684675fa1b\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") "
Sep 15 22:49:22.404955 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.404926    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-envoy\") pod \"89221884-c915-4606-a1ac-3d684675fa1b\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") "
Sep 15 22:49:22.405239 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.404962    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89221884-c915-4606-a1ac-3d684675fa1b-istio-podinfo\") pod \"89221884-c915-4606-a1ac-3d684675fa1b\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") "
Sep 15 22:49:22.405239 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-data\") pod \"89221884-c915-4606-a1ac-3d684675fa1b\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") "
Sep 15 22:49:22.405239 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-istio-token\") pod \"89221884-c915-4606-a1ac-3d684675fa1b\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") "
Sep 15 22:49:22.405239 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405068    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-socket\") pod \"89221884-c915-4606-a1ac-3d684675fa1b\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") "
Sep 15 22:49:22.405239 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405087    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-credential-socket\") pod \"89221884-c915-4606-a1ac-3d684675fa1b\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") "
Sep 15 22:49:22.405239 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405142    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89221884-c915-4606-a1ac-3d684675fa1b-istiod-ca-cert\") pod \"89221884-c915-4606-a1ac-3d684675fa1b\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") "
Sep 15 22:49:22.405239 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405212    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "89221884-c915-4606-a1ac-3d684675fa1b" (UID: "89221884-c915-4606-a1ac-3d684675fa1b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:49:22.405593 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405225    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbnvx\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-kube-api-access-bbnvx\") pod \"89221884-c915-4606-a1ac-3d684675fa1b\" (UID: \"89221884-c915-4606-a1ac-3d684675fa1b\") "
Sep 15 22:49:22.405593 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405456    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-data" (OuterVolumeSpecName: "istio-data") pod "89221884-c915-4606-a1ac-3d684675fa1b" (UID: "89221884-c915-4606-a1ac-3d684675fa1b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:49:22.405593 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405474    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "89221884-c915-4606-a1ac-3d684675fa1b" (UID: "89221884-c915-4606-a1ac-3d684675fa1b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:49:22.405757 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405601    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "89221884-c915-4606-a1ac-3d684675fa1b" (UID: "89221884-c915-4606-a1ac-3d684675fa1b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:49:22.405819 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405766    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89221884-c915-4606-a1ac-3d684675fa1b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "89221884-c915-4606-a1ac-3d684675fa1b" (UID: "89221884-c915-4606-a1ac-3d684675fa1b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:49:22.405819 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405771    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:22.405819 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405814    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:22.405940 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405831    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:22.405940 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.405845    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:22.407473 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.407443    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-kube-api-access-bbnvx" (OuterVolumeSpecName: "kube-api-access-bbnvx") pod "89221884-c915-4606-a1ac-3d684675fa1b" (UID: "89221884-c915-4606-a1ac-3d684675fa1b"). InnerVolumeSpecName "kube-api-access-bbnvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:49:22.407589 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.407472    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-istio-token" (OuterVolumeSpecName: "istio-token") pod "89221884-c915-4606-a1ac-3d684675fa1b" (UID: "89221884-c915-4606-a1ac-3d684675fa1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:49:22.407649 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.407613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89221884-c915-4606-a1ac-3d684675fa1b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "89221884-c915-4606-a1ac-3d684675fa1b" (UID: "89221884-c915-4606-a1ac-3d684675fa1b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:49:22.407649 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.407612    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "89221884-c915-4606-a1ac-3d684675fa1b" (UID: "89221884-c915-4606-a1ac-3d684675fa1b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:49:22.506800 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.506760    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89221884-c915-4606-a1ac-3d684675fa1b-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:22.506800 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.506789    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbnvx\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-kube-api-access-bbnvx\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:22.506800 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.506800    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89221884-c915-4606-a1ac-3d684675fa1b-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:22.506800 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.506809    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89221884-c915-4606-a1ac-3d684675fa1b-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:22.507081 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.506817    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89221884-c915-4606-a1ac-3d684675fa1b-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:49:22.941354 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.941322    2578 generic.go:356] "Generic (PLEG): container finished" podID="89221884-c915-4606-a1ac-3d684675fa1b" containerID="d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324" exitCode=0
Sep 15 22:49:22.941560 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.941378    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"
Sep 15 22:49:22.941560 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.941402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" event={"ID":"89221884-c915-4606-a1ac-3d684675fa1b","Type":"ContainerDied","Data":"d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324"}
Sep 15 22:49:22.941560 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.941459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4" event={"ID":"89221884-c915-4606-a1ac-3d684675fa1b","Type":"ContainerDied","Data":"df730a30728a9b4ead87fed0cb2447ae26af0e5a1e5c1224d9f0752def195583"}
Sep 15 22:49:22.941560 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.941481    2578 scope.go:117] "RemoveContainer" containerID="d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324"
Sep 15 22:49:22.950575 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.950554    2578 scope.go:117] "RemoveContainer" containerID="9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa"
Sep 15 22:49:22.959869 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.959844    2578 scope.go:117] "RemoveContainer" containerID="d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324"
Sep 15 22:49:22.960306 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:49:22.960272    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324\": container with ID starting with d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324 not found: ID does not exist" containerID="d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324"
Sep 15 22:49:22.960424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.960318    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324"} err="failed to get container status \"d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324\": rpc error: code = NotFound desc = could not find container \"d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324\": container with ID starting with d882834edce529d001885f211f481f47a7a2b10fc438511361e6a2d8af447324 not found: ID does not exist"
Sep 15 22:49:22.960424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.960349    2578 scope.go:117] "RemoveContainer" containerID="9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa"
Sep 15 22:49:22.960556 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.960524    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"]
Sep 15 22:49:22.960832 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:49:22.960770    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa\": container with ID starting with 9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa not found: ID does not exist" containerID="9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa"
Sep 15 22:49:22.960832 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.960807    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa"} err="failed to get container status \"9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa\": rpc error: code = NotFound desc = could not find container \"9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa\": container with ID starting with 9fa827ccf220eef0c854864a6ecd1ff2c9856e6f7137414a6613666efc28a0fa not found: ID does not exist"
Sep 15 22:49:22.963134 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:22.963113    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-765e6d-openshift-default-874566f4b-699h4"]
Sep 15 22:49:24.763839 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:24.763802    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89221884-c915-4606-a1ac-3d684675fa1b" path="/var/lib/kubelet/pods/89221884-c915-4606-a1ac-3d684675fa1b/volumes"
Sep 15 22:49:48.068213 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:48.068175    2578 generic.go:356] "Generic (PLEG): container finished" podID="09e12005-9215-41e3-8808-c54d4bfca288" containerID="32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe" exitCode=0
Sep 15 22:49:48.068613 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:48.068229    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" event={"ID":"09e12005-9215-41e3-8808-c54d4bfca288","Type":"ContainerDied","Data":"32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe"}
Sep 15 22:49:48.068719 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:48.068703    2578 scope.go:117] "RemoveContainer" containerID="32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe"
Sep 15 22:49:48.069826 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:48.069796    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:49:48.069891 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:48.069866    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:49:48.069927 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:48.069895    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:49:49.075251 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:49.075212    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" event={"ID":"09e12005-9215-41e3-8808-c54d4bfca288","Type":"ContainerStarted","Data":"26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b"}
Sep 15 22:49:49.141518 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:49.141486    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:49.142653 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:49.142628    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 22:49:49.142785 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:49.142678    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 22:49:49.831535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:49.831493    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"]
Sep 15 22:49:49.923661 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:49.923125    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:50.141550 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:50.141466    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 22:49:50.141550 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:50.141518    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 22:49:51.141847 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.141804    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 22:49:51.142238 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.141886    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 22:49:51.523624 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.523579    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"]
Sep 15 22:49:51.524234 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.524210    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy"
Sep 15 22:49:51.524234 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.524236    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy"
Sep 15 22:49:51.524432 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.524260    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy"
Sep 15 22:49:51.524432 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.524268    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy"
Sep 15 22:49:51.524432 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.524372    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy"
Sep 15 22:49:51.524432 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.524386    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="89221884-c915-4606-a1ac-3d684675fa1b" containerName="istio-proxy"
Sep 15 22:49:51.526747 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.526722    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.529457 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.529434    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-92482b-openshift-default-dockercfg-cpnqc\""
Sep 15 22:49:51.544672 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.544642    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"]
Sep 15 22:49:51.631118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.630146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-socket\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.631118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.630215    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-credential-socket\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.631118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.630258    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-podinfo\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.631118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.630294    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-token\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.631118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.630324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-envoy\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.631118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.630400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wlng\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-kube-api-access-4wlng\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.631118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.630426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-data\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.631118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.630518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-certs\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.631118 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.630545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istiod-ca-cert\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731071 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-socket\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731071 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-credential-socket\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731130    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-podinfo\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-token\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-envoy\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wlng\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-kube-api-access-4wlng\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731243    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-data\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731325    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-certs\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istiod-ca-cert\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731732 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731490    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-socket\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731732 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731672    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-credential-socket\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.731880 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.731857    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-data\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.732155 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.732132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-certs\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.732278 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.732260    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istiod-ca-cert\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.733837 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.733813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-podinfo\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.734200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.734178    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-envoy\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.739897 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.739872    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wlng\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-kube-api-access-4wlng\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.740222 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.740205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-token\") pod \"e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") " pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:51.837927 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:51.837818    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:52.136594 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:52.136563    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"]
Sep 15 22:49:52.138016 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:49:52.137983    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be7eeef_fc8e_4b9f_b1c1_aa045d2872c2.slice/crio-ea599e713bc7055f786124b66a678b5da83493f5e730f31345517de1c8ae479d WatchSource:0}: Error finding container ea599e713bc7055f786124b66a678b5da83493f5e730f31345517de1c8ae479d: Status 404 returned error can't find the container with id ea599e713bc7055f786124b66a678b5da83493f5e730f31345517de1c8ae479d
Sep 15 22:49:52.140820 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:52.140781    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:49:52.140914 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:52.140857    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:49:52.140914 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:52.140885    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:49:52.141480 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:52.141461    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 22:49:52.141558 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:52.141509    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 22:49:53.099748 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:53.099697    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" event={"ID":"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2","Type":"ContainerStarted","Data":"c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634"}
Sep 15 22:49:53.100286 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:53.099777    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" event={"ID":"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2","Type":"ContainerStarted","Data":"ea599e713bc7055f786124b66a678b5da83493f5e730f31345517de1c8ae479d"}
Sep 15 22:49:53.125020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:53.124949    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" podStartSLOduration=2.124926627 podStartE2EDuration="2.124926627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:49:53.124512163 +0000 UTC m=+1332.878136292" watchObservedRunningTime="2026-09-15 22:49:53.124926627 +0000 UTC m=+1332.878550755"
Sep 15 22:49:53.141863 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:53.141830    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 22:49:53.142039 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:53.141885    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 22:49:53.838812 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:53.838768    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:53.844062 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:53.844035    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:54.105690 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:54.105581    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:54.106776 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:54.106752    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:49:54.141636 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:54.141446    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 22:49:54.141636 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:54.141516    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 22:49:55.145839 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:55.145786    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:55.147577 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:55.147554    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 22:49:59.262355 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.262272    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"]
Sep 15 22:49:59.264929 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.264905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.267725 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.267704    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-92482b-serving-cert\""
Sep 15 22:49:59.277876 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.277841    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"]
Sep 15 22:49:59.307364 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.307331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a635b117-00e1-4357-a8ef-e2a0caf593d8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.307535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.307374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.307535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.307407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k797n\" (UniqueName: \"kubernetes.io/projected/a635b117-00e1-4357-a8ef-e2a0caf593d8-kube-api-access-k797n\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.307535 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.307468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.408464 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.408427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k797n\" (UniqueName: \"kubernetes.io/projected/a635b117-00e1-4357-a8ef-e2a0caf593d8-kube-api-access-k797n\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.408647 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.408491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.408647 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.408576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a635b117-00e1-4357-a8ef-e2a0caf593d8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.408647 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.408601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.408760 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:49:59.408660    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-serving-cert: secret "maas-api-e2e-shared-b-w3-92482b-serving-cert" not found
Sep 15 22:49:59.408760 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:49:59.408744    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-maas-api-tls podName:a635b117-00e1-4357-a8ef-e2a0caf593d8 nodeName:}" failed. No retries permitted until 2026-09-15 22:49:59.908720976 +0000 UTC m=+1339.662345092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-maas-api-tls") pod "maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht" (UID: "a635b117-00e1-4357-a8ef-e2a0caf593d8") : secret "maas-api-e2e-shared-b-w3-92482b-serving-cert" not found
Sep 15 22:49:59.409315 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.409291    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a635b117-00e1-4357-a8ef-e2a0caf593d8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.411238 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.411215    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.417210 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.417189    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k797n\" (UniqueName: \"kubernetes.io/projected/a635b117-00e1-4357-a8ef-e2a0caf593d8-kube-api-access-k797n\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.914154 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.914083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:49:59.916860 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:49:59.916832    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:50:00.178049 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:00.177949    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:50:00.448890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:00.448858    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"]
Sep 15 22:50:00.451503 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:50:00.451476    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda635b117_00e1_4357_a8ef_e2a0caf593d8.slice/crio-876b2d1c3daa9d336c051f620afa14dc4227ff7e8b917f1c38c57547fedd3ddd WatchSource:0}: Error finding container 876b2d1c3daa9d336c051f620afa14dc4227ff7e8b917f1c38c57547fedd3ddd: Status 404 returned error can't find the container with id 876b2d1c3daa9d336c051f620afa14dc4227ff7e8b917f1c38c57547fedd3ddd
Sep 15 22:50:00.453077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:00.453055    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:50:01.147708 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:01.147601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht" event={"ID":"a635b117-00e1-4357-a8ef-e2a0caf593d8","Type":"ContainerStarted","Data":"cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84"}
Sep 15 22:50:01.147708 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:01.147665    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht" event={"ID":"a635b117-00e1-4357-a8ef-e2a0caf593d8","Type":"ContainerStarted","Data":"876b2d1c3daa9d336c051f620afa14dc4227ff7e8b917f1c38c57547fedd3ddd"}
Sep 15 22:50:01.147708 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:01.147693    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:50:01.168875 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:01.168820    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht" podStartSLOduration=1.852315993 podStartE2EDuration="2.168805679s" totalImagesPullingTime="316.489686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:50:01.167507799 +0000 UTC m=+1340.921131926" watchObservedRunningTime="2026-09-15 22:50:01.168805679 +0000 UTC m=+1340.922429805"
Sep 15 22:50:04.851375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:04.851282    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb" podUID="68cb1996-affd-45d3-880f-8505fa1564ea" containerName="main" containerID="cri-o://8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad" gracePeriod=45
Sep 15 22:50:05.100687 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.100656    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:50:05.170235 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.170140    2578 generic.go:356] "Generic (PLEG): container finished" podID="68cb1996-affd-45d3-880f-8505fa1564ea" containerID="8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad" exitCode=0
Sep 15 22:50:05.170235 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.170185    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"
Sep 15 22:50:05.170422 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.170227    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb" event={"ID":"68cb1996-affd-45d3-880f-8505fa1564ea","Type":"ContainerDied","Data":"8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad"}
Sep 15 22:50:05.170422 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.170273    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb" event={"ID":"68cb1996-affd-45d3-880f-8505fa1564ea","Type":"ContainerDied","Data":"15f052be1697f25dda156e289efa3ca66e99ca310c5bd67814e4f62c9bbbb68e"}
Sep 15 22:50:05.170422 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.170294    2578 scope.go:117] "RemoveContainer" containerID="8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad"
Sep 15 22:50:05.178631 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.178608    2578 scope.go:117] "RemoveContainer" containerID="8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad"
Sep 15 22:50:05.178940 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:50:05.178920    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad\": container with ID starting with 8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad not found: ID does not exist" containerID="8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad"
Sep 15 22:50:05.178979 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.178951    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad"} err="failed to get container status \"8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad\": rpc error: code = NotFound desc = could not find container \"8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad\": container with ID starting with 8461d3a97606a30360720a7c4d82363a07fc13367855ced8b28a85591d4ab7ad not found: ID does not exist"
Sep 15 22:50:05.181302 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181282    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cb1996-affd-45d3-880f-8505fa1564ea-tls-certs\") pod \"68cb1996-affd-45d3-880f-8505fa1564ea\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") "
Sep 15 22:50:05.181362 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181318    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-tmp-dir\") pod \"68cb1996-affd-45d3-880f-8505fa1564ea\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") "
Sep 15 22:50:05.181362 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181342    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-model-cache\") pod \"68cb1996-affd-45d3-880f-8505fa1564ea\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") "
Sep 15 22:50:05.181443 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181394    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-home\") pod \"68cb1996-affd-45d3-880f-8505fa1564ea\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") "
Sep 15 22:50:05.181443 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181436    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcmlt\" (UniqueName: \"kubernetes.io/projected/68cb1996-affd-45d3-880f-8505fa1564ea-kube-api-access-kcmlt\") pod \"68cb1996-affd-45d3-880f-8505fa1564ea\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") "
Sep 15 22:50:05.181553 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181471    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-dshm\") pod \"68cb1996-affd-45d3-880f-8505fa1564ea\" (UID: \"68cb1996-affd-45d3-880f-8505fa1564ea\") "
Sep 15 22:50:05.181553 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181538    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "68cb1996-affd-45d3-880f-8505fa1564ea" (UID: "68cb1996-affd-45d3-880f-8505fa1564ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:50:05.181681 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181646    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-model-cache" (OuterVolumeSpecName: "model-cache") pod "68cb1996-affd-45d3-880f-8505fa1564ea" (UID: "68cb1996-affd-45d3-880f-8505fa1564ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:50:05.181681 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181659    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-home" (OuterVolumeSpecName: "home") pod "68cb1996-affd-45d3-880f-8505fa1564ea" (UID: "68cb1996-affd-45d3-880f-8505fa1564ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:50:05.181941 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181923    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:05.182012 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181945    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:05.182012 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.181957    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:05.183574 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.183551    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-dshm" (OuterVolumeSpecName: "dshm") pod "68cb1996-affd-45d3-880f-8505fa1564ea" (UID: "68cb1996-affd-45d3-880f-8505fa1564ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:50:05.183668 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.183599    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68cb1996-affd-45d3-880f-8505fa1564ea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "68cb1996-affd-45d3-880f-8505fa1564ea" (UID: "68cb1996-affd-45d3-880f-8505fa1564ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:50:05.183668 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.183621    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68cb1996-affd-45d3-880f-8505fa1564ea-kube-api-access-kcmlt" (OuterVolumeSpecName: "kube-api-access-kcmlt") pod "68cb1996-affd-45d3-880f-8505fa1564ea" (UID: "68cb1996-affd-45d3-880f-8505fa1564ea"). InnerVolumeSpecName "kube-api-access-kcmlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:50:05.282566 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.282530    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cb1996-affd-45d3-880f-8505fa1564ea-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:05.282566 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.282559    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcmlt\" (UniqueName: \"kubernetes.io/projected/68cb1996-affd-45d3-880f-8505fa1564ea-kube-api-access-kcmlt\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:05.282566 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.282569    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cb1996-affd-45d3-880f-8505fa1564ea-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:05.496293 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.496258    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"]
Sep 15 22:50:05.498126 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:05.498088    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/e2e-conflict-b-51df66af-kserve-867677db86-jfngb"]
Sep 15 22:50:06.767907 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:06.767868    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68cb1996-affd-45d3-880f-8505fa1564ea" path="/var/lib/kubelet/pods/68cb1996-affd-45d3-880f-8505fa1564ea/volumes"
Sep 15 22:50:07.158869 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:07.158790    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 22:50:10.054648 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:10.054610    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"]
Sep 15 22:50:25.074323 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.074217    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z" podUID="89c7d92b-1c77-405c-b890-7eaf0474ed67" containerName="main" containerID="cri-o://dccdac54fcfdf1924b79a4db2b145073a01e1d1e8b1403bc41b21db1c67b9791" gracePeriod=45
Sep 15 22:50:25.283534 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.283497    2578 generic.go:356] "Generic (PLEG): container finished" podID="89c7d92b-1c77-405c-b890-7eaf0474ed67" containerID="dccdac54fcfdf1924b79a4db2b145073a01e1d1e8b1403bc41b21db1c67b9791" exitCode=0
Sep 15 22:50:25.283707 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.283590    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z" event={"ID":"89c7d92b-1c77-405c-b890-7eaf0474ed67","Type":"ContainerDied","Data":"dccdac54fcfdf1924b79a4db2b145073a01e1d1e8b1403bc41b21db1c67b9791"}
Sep 15 22:50:25.337996 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.337924    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:50:25.392087 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.392057    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-model-cache\") pod \"89c7d92b-1c77-405c-b890-7eaf0474ed67\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") "
Sep 15 22:50:25.392303 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.392134    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89c7d92b-1c77-405c-b890-7eaf0474ed67-tls-certs\") pod \"89c7d92b-1c77-405c-b890-7eaf0474ed67\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") "
Sep 15 22:50:25.392303 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.392262    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcfnx\" (UniqueName: \"kubernetes.io/projected/89c7d92b-1c77-405c-b890-7eaf0474ed67-kube-api-access-rcfnx\") pod \"89c7d92b-1c77-405c-b890-7eaf0474ed67\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") "
Sep 15 22:50:25.392303 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.392296    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-tmp-dir\") pod \"89c7d92b-1c77-405c-b890-7eaf0474ed67\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") "
Sep 15 22:50:25.392481 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.392328    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-dshm\") pod \"89c7d92b-1c77-405c-b890-7eaf0474ed67\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") "
Sep 15 22:50:25.392481 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.392341    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-model-cache" (OuterVolumeSpecName: "model-cache") pod "89c7d92b-1c77-405c-b890-7eaf0474ed67" (UID: "89c7d92b-1c77-405c-b890-7eaf0474ed67"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:50:25.392481 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.392383    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-home\") pod \"89c7d92b-1c77-405c-b890-7eaf0474ed67\" (UID: \"89c7d92b-1c77-405c-b890-7eaf0474ed67\") "
Sep 15 22:50:25.392657 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.392507    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "89c7d92b-1c77-405c-b890-7eaf0474ed67" (UID: "89c7d92b-1c77-405c-b890-7eaf0474ed67"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:50:25.393059 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.392998    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:25.393059 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.393029    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:25.393388 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.393351    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-home" (OuterVolumeSpecName: "home") pod "89c7d92b-1c77-405c-b890-7eaf0474ed67" (UID: "89c7d92b-1c77-405c-b890-7eaf0474ed67"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:50:25.395157 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.395131    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89c7d92b-1c77-405c-b890-7eaf0474ed67-kube-api-access-rcfnx" (OuterVolumeSpecName: "kube-api-access-rcfnx") pod "89c7d92b-1c77-405c-b890-7eaf0474ed67" (UID: "89c7d92b-1c77-405c-b890-7eaf0474ed67"). InnerVolumeSpecName "kube-api-access-rcfnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:50:25.395331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.395300    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89c7d92b-1c77-405c-b890-7eaf0474ed67-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "89c7d92b-1c77-405c-b890-7eaf0474ed67" (UID: "89c7d92b-1c77-405c-b890-7eaf0474ed67"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:50:25.395415 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.395380    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-dshm" (OuterVolumeSpecName: "dshm") pod "89c7d92b-1c77-405c-b890-7eaf0474ed67" (UID: "89c7d92b-1c77-405c-b890-7eaf0474ed67"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:50:25.494058 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.494015    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcfnx\" (UniqueName: \"kubernetes.io/projected/89c7d92b-1c77-405c-b890-7eaf0474ed67-kube-api-access-rcfnx\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:25.494058 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.494054    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:25.494058 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.494064    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89c7d92b-1c77-405c-b890-7eaf0474ed67-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:25.494326 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:25.494071    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89c7d92b-1c77-405c-b890-7eaf0474ed67-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:50:26.291583 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:26.291556    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"
Sep 15 22:50:26.292006 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:26.291549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z" event={"ID":"89c7d92b-1c77-405c-b890-7eaf0474ed67","Type":"ContainerDied","Data":"60d6e34b568879cd060c615efd96029b6a78308db278219955826da901cf30d8"}
Sep 15 22:50:26.292006 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:26.291685    2578 scope.go:117] "RemoveContainer" containerID="dccdac54fcfdf1924b79a4db2b145073a01e1d1e8b1403bc41b21db1c67b9791"
Sep 15 22:50:26.325051 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:26.325019    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"]
Sep 15 22:50:26.329465 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:26.329439    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6cdcda22/e2e-lifecycle-model-6cdcda22-kserve-5f7d7ff57f-j829z"]
Sep 15 22:50:26.763124 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:26.763080    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89c7d92b-1c77-405c-b890-7eaf0474ed67" path="/var/lib/kubelet/pods/89c7d92b-1c77-405c-b890-7eaf0474ed67/volumes"
Sep 15 22:50:28.304712 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:28.304679    2578 generic.go:356] "Generic (PLEG): container finished" podID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerID="c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634" exitCode=0
Sep 15 22:50:28.305162 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:28.304751    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" event={"ID":"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2","Type":"ContainerDied","Data":"c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634"}
Sep 15 22:50:28.305212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:28.305184    2578 scope.go:117] "RemoveContainer" containerID="c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634"
Sep 15 22:50:28.306405 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:28.306377    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:50:28.306478 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:28.306444    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:50:28.306516 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:28.306476    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:50:29.315334 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:29.315298    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" event={"ID":"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2","Type":"ContainerStarted","Data":"02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b"}
Sep 15 22:50:29.839002 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:29.838962    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:50:29.839328 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:29.839303    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 22:50:29.839418 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:29.839360    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 22:50:30.838873 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:30.838827    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 22:50:30.839272 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:30.838892    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 22:50:31.838984 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:31.838939    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 22:50:31.839510 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:31.839012    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 22:50:32.839278 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:32.839235    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 22:50:32.839674 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:32.839306    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 22:50:33.838634 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:33.838601    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 22:50:33.838813 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:33.838657    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 22:50:34.842788 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:34.842750    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:50:34.843270 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:34.843039    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:50:34.843906 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:34.843885    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 22:50:56.856140 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.856088    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"]
Sep 15 22:50:56.856746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.856703    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c7d92b-1c77-405c-b890-7eaf0474ed67" containerName="main"
Sep 15 22:50:56.856746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.856724    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c7d92b-1c77-405c-b890-7eaf0474ed67" containerName="main"
Sep 15 22:50:56.856746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.856736    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68cb1996-affd-45d3-880f-8505fa1564ea" containerName="main"
Sep 15 22:50:56.856746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.856744    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="68cb1996-affd-45d3-880f-8505fa1564ea" containerName="main"
Sep 15 22:50:56.856968 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.856856    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="68cb1996-affd-45d3-880f-8505fa1564ea" containerName="main"
Sep 15 22:50:56.856968 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.856871    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c7d92b-1c77-405c-b890-7eaf0474ed67" containerName="main"
Sep 15 22:50:56.859798 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.859775    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:56.862659 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.862638    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0016d4\"/\"kube-root-ca.crt\""
Sep 15 22:50:56.864192 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.864172    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0016d4\"/\"openshift-service-ca.crt\""
Sep 15 22:50:56.864192 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.864185    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0016d4\"/\"auth-test-model-0016d4-kserve-self-signed-certs\""
Sep 15 22:50:56.864345 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.864172    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0016d4\"/\"default-dockercfg-nhjlf\""
Sep 15 22:50:56.870484 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:56.870460    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"]
Sep 15 22:50:57.013709 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.013664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-home\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.013934 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.013730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-model-cache\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.013934 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.013810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tls-certs\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.013934 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.013848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbtjq\" (UniqueName: \"kubernetes.io/projected/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-kube-api-access-dbtjq\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.013934 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.013929    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tmp-dir\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.014150 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.013954    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-dshm\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.115089 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.114980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbtjq\" (UniqueName: \"kubernetes.io/projected/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-kube-api-access-dbtjq\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.115089 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.115055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tmp-dir\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.115089 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.115084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-dshm\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.115437 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.115185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-home\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.115437 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.115217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-model-cache\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.115437 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.115345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tls-certs\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.115583 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.115556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tmp-dir\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.115644 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.115594    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-model-cache\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.115704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.115654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-home\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.117570 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.117540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-dshm\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.117772 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.117756    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tls-certs\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.124157 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.124120    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbtjq\" (UniqueName: \"kubernetes.io/projected/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-kube-api-access-dbtjq\") pod \"auth-test-model-0016d4-kserve-596496bdcf-6sl8d\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.170244 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.170202    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:50:57.441561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.441532    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"]
Sep 15 22:50:57.443707 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:50:57.443665    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50f22a4a_bab3_4c6f_beb4_c2a4f7246d6b.slice/crio-c232908c92e4a3bd46a8e17a05b656e8b4c2a7eb53e5e536b64bc1409e3e188a WatchSource:0}: Error finding container c232908c92e4a3bd46a8e17a05b656e8b4c2a7eb53e5e536b64bc1409e3e188a: Status 404 returned error can't find the container with id c232908c92e4a3bd46a8e17a05b656e8b4c2a7eb53e5e536b64bc1409e3e188a
Sep 15 22:50:57.475552 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:57.475513    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d" event={"ID":"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b","Type":"ContainerStarted","Data":"c232908c92e4a3bd46a8e17a05b656e8b4c2a7eb53e5e536b64bc1409e3e188a"}
Sep 15 22:50:58.482205 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:58.482165    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d" event={"ID":"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b","Type":"ContainerStarted","Data":"22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc"}
Sep 15 22:50:58.500832 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:50:58.500768    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d" podStartSLOduration=2.500750184 podStartE2EDuration="2.500750184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:50:58.498760822 +0000 UTC m=+1398.252384949" watchObservedRunningTime="2026-09-15 22:50:58.500750184 +0000 UTC m=+1398.254374311"
Sep 15 22:51:07.172065 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:51:07.172028    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:51:07.172768 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:51:07.172740    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:51:07.190078 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:51:07.190046    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:51:07.537208 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:51:07.537175    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:52:26.542601 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.542557    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"]
Sep 15 22:52:26.545269 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.545244    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.548169 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.548147    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-fe020d\"/\"embedding-w0-fe020d-kserve-self-signed-certs\""
Sep 15 22:52:26.554225 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.554198    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"]
Sep 15 22:52:26.640290 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.640249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm9pg\" (UniqueName: \"kubernetes.io/projected/7b949c63-0745-43d1-a445-beefb836ad19-kube-api-access-sm9pg\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.640505 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.640367    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b949c63-0745-43d1-a445-beefb836ad19-tls-certs\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.640505 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.640449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-model-cache\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.640505 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.640465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-dshm\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.640662 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.640535    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-home\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.640662 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.640556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-tmp-dir\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.741871 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.741837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-home\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.742046 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.741881    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-tmp-dir\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.742046 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.741931    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm9pg\" (UniqueName: \"kubernetes.io/projected/7b949c63-0745-43d1-a445-beefb836ad19-kube-api-access-sm9pg\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.742046 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.741998    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b949c63-0745-43d1-a445-beefb836ad19-tls-certs\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.742251 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.742077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-model-cache\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.742251 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.742118    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-dshm\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.742364 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.742322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-home\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.742364 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.742345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-tmp-dir\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.742475 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.742452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-model-cache\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.744472 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.744445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-dshm\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.744632 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.744503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b949c63-0745-43d1-a445-beefb836ad19-tls-certs\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.751071 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.751042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm9pg\" (UniqueName: \"kubernetes.io/projected/7b949c63-0745-43d1-a445-beefb836ad19-kube-api-access-sm9pg\") pod \"embedding-w0-fe020d-kserve-5976fc9449-qf52n\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") " pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:26.854973 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:26.854881    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:27.129146 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:27.129012    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"]
Sep 15 22:52:27.132675 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:52:27.132643    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b949c63_0745_43d1_a445_beefb836ad19.slice/crio-8250aa4b02b62a0031c86e635682908c42d3c1007819389f576a557bdbd5d5fd WatchSource:0}: Error finding container 8250aa4b02b62a0031c86e635682908c42d3c1007819389f576a557bdbd5d5fd: Status 404 returned error can't find the container with id 8250aa4b02b62a0031c86e635682908c42d3c1007819389f576a557bdbd5d5fd
Sep 15 22:52:27.955541 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:27.955499    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n" event={"ID":"7b949c63-0745-43d1-a445-beefb836ad19","Type":"ContainerStarted","Data":"00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed"}
Sep 15 22:52:27.955941 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:27.955553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n" event={"ID":"7b949c63-0745-43d1-a445-beefb836ad19","Type":"ContainerStarted","Data":"8250aa4b02b62a0031c86e635682908c42d3c1007819389f576a557bdbd5d5fd"}
Sep 15 22:52:27.973634 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:27.973576    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n" podStartSLOduration=1.973560873 podStartE2EDuration="1.973560873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:52:27.972405826 +0000 UTC m=+1487.726029953" watchObservedRunningTime="2026-09-15 22:52:27.973560873 +0000 UTC m=+1487.727185053"
Sep 15 22:52:35.668845 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.668809    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"]
Sep 15 22:52:35.672545 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.672518    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.675575 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.675552    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f667ee-openshift-default-dockercfg-6rs9g\""
Sep 15 22:52:35.684414 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.684392    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"]
Sep 15 22:52:35.837543 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.837502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b2kd\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-kube-api-access-7b2kd\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.837802 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.837783    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/241305ff-629f-4468-a196-c7b57b0a9470-istiod-ca-cert\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.837938 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.837925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-istio-token\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.838050 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.838038    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-credential-socket\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.838238 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.838224    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-socket\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.838372 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.838360    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-data\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.838473 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.838460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-certs\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.838569 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.838558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-envoy\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.838672 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.838660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/241305ff-629f-4468-a196-c7b57b0a9470-istio-podinfo\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.939870 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.939833    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-certs\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.939870 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.939883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-envoy\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940128 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.939910    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/241305ff-629f-4468-a196-c7b57b0a9470-istio-podinfo\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940128 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.939957    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b2kd\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-kube-api-access-7b2kd\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940128 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.939980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/241305ff-629f-4468-a196-c7b57b0a9470-istiod-ca-cert\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940128 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.940025    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-istio-token\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.940219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-credential-socket\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940331 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.940307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-socket\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940437 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.940399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-data\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940437 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.940419    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-certs\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940657 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.940631    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-credential-socket\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940758 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.940722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-socket\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940758 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.940733    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/241305ff-629f-4468-a196-c7b57b0a9470-istiod-ca-cert\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.940865 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.940788    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-data\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.942544 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.942522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-envoy\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.942667 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.942650    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/241305ff-629f-4468-a196-c7b57b0a9470-istio-podinfo\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.949716 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.949685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b2kd\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-kube-api-access-7b2kd\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.949825 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.949691    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-istio-token\") pod \"e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") " pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:35.984567 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:35.984534    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:36.266676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:36.266643    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"]
Sep 15 22:52:36.268963 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:52:36.268928    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241305ff_629f_4468_a196_c7b57b0a9470.slice/crio-4bd0fa8f97472978027978b2347520cfe8c81d75c588bb9d9045b16741bc7973 WatchSource:0}: Error finding container 4bd0fa8f97472978027978b2347520cfe8c81d75c588bb9d9045b16741bc7973: Status 404 returned error can't find the container with id 4bd0fa8f97472978027978b2347520cfe8c81d75c588bb9d9045b16741bc7973
Sep 15 22:52:36.271435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:36.271397    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:52:36.271525 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:36.271468    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:52:36.271525 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:36.271498    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:52:36.855720 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:36.855677    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:36.856170 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:36.855750    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:36.868489 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:36.868461    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:37.003882 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:37.003828    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" event={"ID":"241305ff-629f-4468-a196-c7b57b0a9470","Type":"ContainerStarted","Data":"06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5"}
Sep 15 22:52:37.004125 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:37.003890    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" event={"ID":"241305ff-629f-4468-a196-c7b57b0a9470","Type":"ContainerStarted","Data":"4bd0fa8f97472978027978b2347520cfe8c81d75c588bb9d9045b16741bc7973"}
Sep 15 22:52:37.015812 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:37.015784    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:52:37.027344 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:37.027294    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" podStartSLOduration=2.027276762 podStartE2EDuration="2.027276762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:52:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:52:37.025230476 +0000 UTC m=+1496.778854603" watchObservedRunningTime="2026-09-15 22:52:37.027276762 +0000 UTC m=+1496.780900889"
Sep 15 22:52:37.985648 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:37.985608    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:37.990951 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:37.990921    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:38.009466 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:38.009429    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:38.010261 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:38.010244    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:52:46.027865 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.027820    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"]
Sep 15 22:52:46.031475 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.031450    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.034379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.034355    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f667ee-serving-cert\""
Sep 15 22:52:46.042309 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.042278    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"]
Sep 15 22:52:46.058972 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.058938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-maas-api-tls\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.059160 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.058993    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-metrics-certs\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.059160 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.059052    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963724ba-d871-4de3-8ed9-958b31fd6c63-openshift-service-ca\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.059248 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.059183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85kpx\" (UniqueName: \"kubernetes.io/projected/963724ba-d871-4de3-8ed9-958b31fd6c63-kube-api-access-85kpx\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.162321 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.162267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85kpx\" (UniqueName: \"kubernetes.io/projected/963724ba-d871-4de3-8ed9-958b31fd6c63-kube-api-access-85kpx\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.162858 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.162827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-maas-api-tls\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.162993 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.162896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-metrics-certs\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.162993 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.162967    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963724ba-d871-4de3-8ed9-958b31fd6c63-openshift-service-ca\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.163893 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.163867    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963724ba-d871-4de3-8ed9-958b31fd6c63-openshift-service-ca\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.168144 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.168112    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-maas-api-tls\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.168290 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.168184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-metrics-certs\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.172515 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.172486    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85kpx\" (UniqueName: \"kubernetes.io/projected/963724ba-d871-4de3-8ed9-958b31fd6c63-kube-api-access-85kpx\") pod \"maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.346257 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.346168    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:46.631759 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:46.631724    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"]
Sep 15 22:52:46.634156 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:52:46.634122    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963724ba_d871_4de3_8ed9_958b31fd6c63.slice/crio-0a0aa8ce54d83a05f71222fa94e9a14211cbe72efe77b064da61949f00ee524c WatchSource:0}: Error finding container 0a0aa8ce54d83a05f71222fa94e9a14211cbe72efe77b064da61949f00ee524c: Status 404 returned error can't find the container with id 0a0aa8ce54d83a05f71222fa94e9a14211cbe72efe77b064da61949f00ee524c
Sep 15 22:52:47.059856 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:47.059810    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf" event={"ID":"963724ba-d871-4de3-8ed9-958b31fd6c63","Type":"ContainerStarted","Data":"e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c"}
Sep 15 22:52:47.060248 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:47.059868    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf" event={"ID":"963724ba-d871-4de3-8ed9-958b31fd6c63","Type":"ContainerStarted","Data":"0a0aa8ce54d83a05f71222fa94e9a14211cbe72efe77b064da61949f00ee524c"}
Sep 15 22:52:47.060248 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:47.059894    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:52:47.101673 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:47.101614    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf" podStartSLOduration=0.756311516 podStartE2EDuration="1.101593517s" totalImagesPullingTime="345.282001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:52:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:52:47.099176963 +0000 UTC m=+1506.852801092" watchObservedRunningTime="2026-09-15 22:52:47.101593517 +0000 UTC m=+1506.855217658"
Sep 15 22:52:53.070624 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:52:53.070591    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:53:21.260319 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:21.260228    2578 generic.go:356] "Generic (PLEG): container finished" podID="241305ff-629f-4468-a196-c7b57b0a9470" containerID="06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5" exitCode=0
Sep 15 22:53:21.260319 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:21.260282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" event={"ID":"241305ff-629f-4468-a196-c7b57b0a9470","Type":"ContainerDied","Data":"06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5"}
Sep 15 22:53:21.260796 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:21.260780    2578 scope.go:117] "RemoveContainer" containerID="06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5"
Sep 15 22:53:21.262210 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:21.262180    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:53:21.262300 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:21.262252    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:53:21.262300 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:21.262286    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:53:22.267875 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:22.267835    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" event={"ID":"241305ff-629f-4468-a196-c7b57b0a9470","Type":"ContainerStarted","Data":"df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e"}
Sep 15 22:53:22.985336 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:22.985297    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:53:22.986434 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:22.986407    2578 patch_prober.go:28] interesting pod/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 22:53:22.986566 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:22.986460    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 22:53:23.009651 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:23.009601    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:53:23.985202 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:23.985167    2578 patch_prober.go:28] interesting pod/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 22:53:23.985620 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:23.985225    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 22:53:24.985743 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:24.985703    2578 patch_prober.go:28] interesting pod/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 22:53:24.986280 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:24.985768    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 22:53:25.985522 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:25.985483    2578 patch_prober.go:28] interesting pod/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 22:53:25.985716 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:25.985544    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 22:53:26.984923 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:26.984881    2578 patch_prober.go:28] interesting pod/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 22:53:26.985364 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:26.984956    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 22:53:27.985600 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:27.985560    2578 patch_prober.go:28] interesting pod/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 22:53:27.986009 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:27.985672    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 22:53:28.989602 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:28.989572    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:53:28.990495 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:28.990475    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:53:50.531571 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.531502    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"]
Sep 15 22:53:50.532284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.532251    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf" podUID="963724ba-d871-4de3-8ed9-958b31fd6c63" containerName="maas-api" containerID="cri-o://e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c" gracePeriod=30
Sep 15 22:53:50.787391 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.787318    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:53:50.904810 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.904769    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963724ba-d871-4de3-8ed9-958b31fd6c63-openshift-service-ca\") pod \"963724ba-d871-4de3-8ed9-958b31fd6c63\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") "
Sep 15 22:53:50.905001 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.904901    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-maas-api-tls\") pod \"963724ba-d871-4de3-8ed9-958b31fd6c63\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") "
Sep 15 22:53:50.905001 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.904950    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-metrics-certs\") pod \"963724ba-d871-4de3-8ed9-958b31fd6c63\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") "
Sep 15 22:53:50.905089 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.905019    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85kpx\" (UniqueName: \"kubernetes.io/projected/963724ba-d871-4de3-8ed9-958b31fd6c63-kube-api-access-85kpx\") pod \"963724ba-d871-4de3-8ed9-958b31fd6c63\" (UID: \"963724ba-d871-4de3-8ed9-958b31fd6c63\") "
Sep 15 22:53:50.905269 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.905243    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/963724ba-d871-4de3-8ed9-958b31fd6c63-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "963724ba-d871-4de3-8ed9-958b31fd6c63" (UID: "963724ba-d871-4de3-8ed9-958b31fd6c63"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:53:50.905575 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.905549    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963724ba-d871-4de3-8ed9-958b31fd6c63-openshift-service-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:53:50.907284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.907255    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "963724ba-d871-4de3-8ed9-958b31fd6c63" (UID: "963724ba-d871-4de3-8ed9-958b31fd6c63"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:53:50.907379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.907255    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/963724ba-d871-4de3-8ed9-958b31fd6c63-kube-api-access-85kpx" (OuterVolumeSpecName: "kube-api-access-85kpx") pod "963724ba-d871-4de3-8ed9-958b31fd6c63" (UID: "963724ba-d871-4de3-8ed9-958b31fd6c63"). InnerVolumeSpecName "kube-api-access-85kpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:53:50.907379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:50.907297    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "963724ba-d871-4de3-8ed9-958b31fd6c63" (UID: "963724ba-d871-4de3-8ed9-958b31fd6c63"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:53:51.007058 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.007012    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-maas-api-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:53:51.007058 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.007059    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963724ba-d871-4de3-8ed9-958b31fd6c63-metrics-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:53:51.007287 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.007077    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85kpx\" (UniqueName: \"kubernetes.io/projected/963724ba-d871-4de3-8ed9-958b31fd6c63-kube-api-access-85kpx\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:53:51.443140 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.443075    2578 generic.go:356] "Generic (PLEG): container finished" podID="963724ba-d871-4de3-8ed9-958b31fd6c63" containerID="e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c" exitCode=0
Sep 15 22:53:51.443140 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.443138    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"
Sep 15 22:53:51.443375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.443169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf" event={"ID":"963724ba-d871-4de3-8ed9-958b31fd6c63","Type":"ContainerDied","Data":"e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c"}
Sep 15 22:53:51.443375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.443232    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf" event={"ID":"963724ba-d871-4de3-8ed9-958b31fd6c63","Type":"ContainerDied","Data":"0a0aa8ce54d83a05f71222fa94e9a14211cbe72efe77b064da61949f00ee524c"}
Sep 15 22:53:51.443375 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.443255    2578 scope.go:117] "RemoveContainer" containerID="e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c"
Sep 15 22:53:51.453036 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.453015    2578 scope.go:117] "RemoveContainer" containerID="e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c"
Sep 15 22:53:51.453369 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:53:51.453352    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c\": container with ID starting with e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c not found: ID does not exist" containerID="e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c"
Sep 15 22:53:51.453447 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.453376    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c"} err="failed to get container status \"e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c\": rpc error: code = NotFound desc = could not find container \"e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c\": container with ID starting with e2d082f3e01526645ef7dac5b7e109bd926b3b3b33377c182b90b7a07efe484c not found: ID does not exist"
Sep 15 22:53:51.468432 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.468405    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"]
Sep 15 22:53:51.473300 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:51.473272    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f667ee-764cbf79f7-5rhhf"]
Sep 15 22:53:52.763353 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:52.763315    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="963724ba-d871-4de3-8ed9-958b31fd6c63" path="/var/lib/kubelet/pods/963724ba-d871-4de3-8ed9-958b31fd6c63/volumes"
Sep 15 22:53:57.230326 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:57.230290    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"]
Sep 15 22:53:57.230738 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:57.230511    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy" containerID="cri-o://df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e" gracePeriod=30
Sep 15 22:53:58.992339 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:58.992301    2578 patch_prober.go:28] interesting pod/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 22:53:58.992748 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:53:58.992357    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 22:54:02.482207 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.482179    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:54:02.505913 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.505314    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"
Sep 15 22:54:02.505913 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.505345    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" event={"ID":"241305ff-629f-4468-a196-c7b57b0a9470","Type":"ContainerDied","Data":"df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e"}
Sep 15 22:54:02.505913 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.505459    2578 scope.go:117] "RemoveContainer" containerID="df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e"
Sep 15 22:54:02.505913 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.505460    2578 generic.go:356] "Generic (PLEG): container finished" podID="241305ff-629f-4468-a196-c7b57b0a9470" containerID="df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e" exitCode=0
Sep 15 22:54:02.505913 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.505561    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9" event={"ID":"241305ff-629f-4468-a196-c7b57b0a9470","Type":"ContainerDied","Data":"4bd0fa8f97472978027978b2347520cfe8c81d75c588bb9d9045b16741bc7973"}
Sep 15 22:54:02.516701 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.516674    2578 scope.go:117] "RemoveContainer" containerID="06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5"
Sep 15 22:54:02.525896 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.525870    2578 scope.go:117] "RemoveContainer" containerID="df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e"
Sep 15 22:54:02.526285 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:02.526263    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e\": container with ID starting with df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e not found: ID does not exist" containerID="df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e"
Sep 15 22:54:02.526439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.526297    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e"} err="failed to get container status \"df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e\": rpc error: code = NotFound desc = could not find container \"df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e\": container with ID starting with df82c1fcd769a57ce3e168d4bc6f6f1d414ff8cad80c43b75551136973ab959e not found: ID does not exist"
Sep 15 22:54:02.526439 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.526323    2578 scope.go:117] "RemoveContainer" containerID="06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5"
Sep 15 22:54:02.526642 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:02.526623    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5\": container with ID starting with 06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5 not found: ID does not exist" containerID="06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5"
Sep 15 22:54:02.526697 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.526647    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5"} err="failed to get container status \"06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5\": rpc error: code = NotFound desc = could not find container \"06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5\": container with ID starting with 06d209a59ae6deaf3ed724f31ee4f4bb2b299a27b670a7b5304187099cb8bab5 not found: ID does not exist"
Sep 15 22:54:02.527974 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.527955    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7b2kd\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-kube-api-access-7b2kd\") pod \"241305ff-629f-4468-a196-c7b57b0a9470\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") "
Sep 15 22:54:02.528040 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.527986    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-socket\") pod \"241305ff-629f-4468-a196-c7b57b0a9470\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") "
Sep 15 22:54:02.528078 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528044    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-envoy\") pod \"241305ff-629f-4468-a196-c7b57b0a9470\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") "
Sep 15 22:54:02.528078 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528074    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-credential-socket\") pod \"241305ff-629f-4468-a196-c7b57b0a9470\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") "
Sep 15 22:54:02.528193 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528128    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-certs\") pod \"241305ff-629f-4468-a196-c7b57b0a9470\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") "
Sep 15 22:54:02.528193 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528160    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/241305ff-629f-4468-a196-c7b57b0a9470-istio-podinfo\") pod \"241305ff-629f-4468-a196-c7b57b0a9470\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") "
Sep 15 22:54:02.528193 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528175    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-data\") pod \"241305ff-629f-4468-a196-c7b57b0a9470\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") "
Sep 15 22:54:02.528352 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528210    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/241305ff-629f-4468-a196-c7b57b0a9470-istiod-ca-cert\") pod \"241305ff-629f-4468-a196-c7b57b0a9470\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") "
Sep 15 22:54:02.528352 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528244    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-istio-token\") pod \"241305ff-629f-4468-a196-c7b57b0a9470\" (UID: \"241305ff-629f-4468-a196-c7b57b0a9470\") "
Sep 15 22:54:02.528453 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528379    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "241305ff-629f-4468-a196-c7b57b0a9470" (UID: "241305ff-629f-4468-a196-c7b57b0a9470"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:02.528750 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528727    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:02.528879 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528803    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-data" (OuterVolumeSpecName: "istio-data") pod "241305ff-629f-4468-a196-c7b57b0a9470" (UID: "241305ff-629f-4468-a196-c7b57b0a9470"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:02.528965 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.528843    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "241305ff-629f-4468-a196-c7b57b0a9470" (UID: "241305ff-629f-4468-a196-c7b57b0a9470"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:02.529235 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.529206    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241305ff-629f-4468-a196-c7b57b0a9470-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "241305ff-629f-4468-a196-c7b57b0a9470" (UID: "241305ff-629f-4468-a196-c7b57b0a9470"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:02.529561 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.529528    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "241305ff-629f-4468-a196-c7b57b0a9470" (UID: "241305ff-629f-4468-a196-c7b57b0a9470"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:02.530599 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.530572    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "241305ff-629f-4468-a196-c7b57b0a9470" (UID: "241305ff-629f-4468-a196-c7b57b0a9470"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:02.530709 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.530604    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-kube-api-access-7b2kd" (OuterVolumeSpecName: "kube-api-access-7b2kd") pod "241305ff-629f-4468-a196-c7b57b0a9470" (UID: "241305ff-629f-4468-a196-c7b57b0a9470"). InnerVolumeSpecName "kube-api-access-7b2kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:02.530833 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.530810    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-istio-token" (OuterVolumeSpecName: "istio-token") pod "241305ff-629f-4468-a196-c7b57b0a9470" (UID: "241305ff-629f-4468-a196-c7b57b0a9470"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:02.530989 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.530972    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/241305ff-629f-4468-a196-c7b57b0a9470-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "241305ff-629f-4468-a196-c7b57b0a9470" (UID: "241305ff-629f-4468-a196-c7b57b0a9470"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:54:02.629664 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.629612    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:02.629664 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.629663    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:02.629664 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.629675    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:02.629664 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.629686    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/241305ff-629f-4468-a196-c7b57b0a9470-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:02.629954 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.629697    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/241305ff-629f-4468-a196-c7b57b0a9470-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:02.629954 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.629708    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/241305ff-629f-4468-a196-c7b57b0a9470-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:02.629954 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.629717    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:02.629954 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.629724    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b2kd\" (UniqueName: \"kubernetes.io/projected/241305ff-629f-4468-a196-c7b57b0a9470-kube-api-access-7b2kd\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:02.823758 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.823722    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"]
Sep 15 22:54:02.829746 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.829717    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f667ee-openshift-default-6977d9548-tpfl9"]
Sep 15 22:54:02.865314 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.865280    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"]
Sep 15 22:54:02.876229 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.876196    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"]
Sep 15 22:54:02.889929 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:02.889890    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"]
Sep 15 22:54:04.764424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:04.764386    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241305ff-629f-4468-a196-c7b57b0a9470" path="/var/lib/kubelet/pods/241305ff-629f-4468-a196-c7b57b0a9470/volumes"
Sep 15 22:54:09.432001 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:09.431962    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"]
Sep 15 22:54:09.432592 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:09.432212    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq" podUID="56267239-c3a7-4494-b01b-c0c73b9fcb1f" containerName="maas-api" containerID="cri-o://9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247" gracePeriod=30
Sep 15 22:54:09.894526 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:09.894498    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:54:10.006538 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.006449    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-metrics-certs\") pod \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") "
Sep 15 22:54:10.006538 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.006506    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4grhs\" (UniqueName: \"kubernetes.io/projected/56267239-c3a7-4494-b01b-c0c73b9fcb1f-kube-api-access-4grhs\") pod \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") "
Sep 15 22:54:10.006538 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.006532    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56267239-c3a7-4494-b01b-c0c73b9fcb1f-openshift-service-ca\") pod \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") "
Sep 15 22:54:10.006788 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.006649    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-maas-api-tls\") pod \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\" (UID: \"56267239-c3a7-4494-b01b-c0c73b9fcb1f\") "
Sep 15 22:54:10.007031 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.006998    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56267239-c3a7-4494-b01b-c0c73b9fcb1f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "56267239-c3a7-4494-b01b-c0c73b9fcb1f" (UID: "56267239-c3a7-4494-b01b-c0c73b9fcb1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:10.008763 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.008734    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "56267239-c3a7-4494-b01b-c0c73b9fcb1f" (UID: "56267239-c3a7-4494-b01b-c0c73b9fcb1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:10.008887 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.008749    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "56267239-c3a7-4494-b01b-c0c73b9fcb1f" (UID: "56267239-c3a7-4494-b01b-c0c73b9fcb1f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:10.008887 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.008797    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56267239-c3a7-4494-b01b-c0c73b9fcb1f-kube-api-access-4grhs" (OuterVolumeSpecName: "kube-api-access-4grhs") pod "56267239-c3a7-4494-b01b-c0c73b9fcb1f" (UID: "56267239-c3a7-4494-b01b-c0c73b9fcb1f"). InnerVolumeSpecName "kube-api-access-4grhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:10.107989 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.107948    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56267239-c3a7-4494-b01b-c0c73b9fcb1f-openshift-service-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:10.107989 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.107983    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-maas-api-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:10.107989 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.107996    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56267239-c3a7-4494-b01b-c0c73b9fcb1f-metrics-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:10.108284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.108007    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4grhs\" (UniqueName: \"kubernetes.io/projected/56267239-c3a7-4494-b01b-c0c73b9fcb1f-kube-api-access-4grhs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:10.554312 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.554274    2578 generic.go:356] "Generic (PLEG): container finished" podID="56267239-c3a7-4494-b01b-c0c73b9fcb1f" containerID="9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247" exitCode=0
Sep 15 22:54:10.554719 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.554320    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"
Sep 15 22:54:10.554719 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.554336    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq" event={"ID":"56267239-c3a7-4494-b01b-c0c73b9fcb1f","Type":"ContainerDied","Data":"9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247"}
Sep 15 22:54:10.554719 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.554386    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq" event={"ID":"56267239-c3a7-4494-b01b-c0c73b9fcb1f","Type":"ContainerDied","Data":"2311794286b374393e50c6b6ff8a299226f2a401f99bb719c3b3b19d2e277332"}
Sep 15 22:54:10.554719 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.554410    2578 scope.go:117] "RemoveContainer" containerID="9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247"
Sep 15 22:54:10.563941 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.563923    2578 scope.go:117] "RemoveContainer" containerID="9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247"
Sep 15 22:54:10.564249 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:10.564226    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247\": container with ID starting with 9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247 not found: ID does not exist" containerID="9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247"
Sep 15 22:54:10.564296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.564260    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247"} err="failed to get container status \"9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247\": rpc error: code = NotFound desc = could not find container \"9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247\": container with ID starting with 9a7c023d4f9b14da6025aad0c94218967a4877db958cdd6a4f6c677ec04bf247 not found: ID does not exist"
Sep 15 22:54:10.579078 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.579042    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"]
Sep 15 22:54:10.584825 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.584796    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-fe020d-7856bb6ff4-vbhcq"]
Sep 15 22:54:10.763366 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:10.763331    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56267239-c3a7-4494-b01b-c0c73b9fcb1f" path="/var/lib/kubelet/pods/56267239-c3a7-4494-b01b-c0c73b9fcb1f/volumes"
Sep 15 22:54:15.187894 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:15.187862    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"]
Sep 15 22:54:15.188407 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:15.188107    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy" containerID="cri-o://7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83" gracePeriod=30
Sep 15 22:54:17.885753 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:17.885685    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n" podUID="7b949c63-0745-43d1-a445-beefb836ad19" containerName="main" containerID="cri-o://00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed" gracePeriod=45
Sep 15 22:54:17.897260 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:17.897225    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb" podUID="0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" containerName="main" containerID="cri-o://4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668" gracePeriod=45
Sep 15 22:54:17.909752 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:17.909716    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz" podUID="3b155b8f-b07e-4801-984f-3609f6bbfc00" containerName="main" containerID="cri-o://ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c" gracePeriod=45
Sep 15 22:54:18.189755 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.189723    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:54:18.221843 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.221813    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:54:18.225554 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.225526    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:54:18.296822 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.296793    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-model-cache\") pod \"3b155b8f-b07e-4801-984f-3609f6bbfc00\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") "
Sep 15 22:54:18.297007 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.296834    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-dshm\") pod \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") "
Sep 15 22:54:18.297007 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.296874    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh6l7\" (UniqueName: \"kubernetes.io/projected/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-kube-api-access-zh6l7\") pod \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") "
Sep 15 22:54:18.297007 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.296901    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm9pg\" (UniqueName: \"kubernetes.io/projected/7b949c63-0745-43d1-a445-beefb836ad19-kube-api-access-sm9pg\") pod \"7b949c63-0745-43d1-a445-beefb836ad19\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") "
Sep 15 22:54:18.297007 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.296960    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b155b8f-b07e-4801-984f-3609f6bbfc00-tls-certs\") pod \"3b155b8f-b07e-4801-984f-3609f6bbfc00\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") "
Sep 15 22:54:18.297007 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.296990    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-dshm\") pod \"7b949c63-0745-43d1-a445-beefb836ad19\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") "
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297024    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tmp-dir\") pod \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") "
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297054    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-home\") pod \"7b949c63-0745-43d1-a445-beefb836ad19\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") "
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297089    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-tmp-dir\") pod \"7b949c63-0745-43d1-a445-beefb836ad19\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") "
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297142    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-home\") pod \"3b155b8f-b07e-4801-984f-3609f6bbfc00\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") "
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297171    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tslpl\" (UniqueName: \"kubernetes.io/projected/3b155b8f-b07e-4801-984f-3609f6bbfc00-kube-api-access-tslpl\") pod \"3b155b8f-b07e-4801-984f-3609f6bbfc00\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") "
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297196    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b949c63-0745-43d1-a445-beefb836ad19-tls-certs\") pod \"7b949c63-0745-43d1-a445-beefb836ad19\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") "
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297213    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b155b8f-b07e-4801-984f-3609f6bbfc00" (UID: "3b155b8f-b07e-4801-984f-3609f6bbfc00"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297224    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-home\") pod \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") "
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297302    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-model-cache\") pod \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") "
Sep 15 22:54:18.297381 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297377    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tls-certs\") pod \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\" (UID: \"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f\") "
Sep 15 22:54:18.297890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297390    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-home" (OuterVolumeSpecName: "home") pod "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" (UID: "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.297890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297444    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-dshm\") pod \"3b155b8f-b07e-4801-984f-3609f6bbfc00\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") "
Sep 15 22:54:18.297890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297484    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-model-cache\") pod \"7b949c63-0745-43d1-a445-beefb836ad19\" (UID: \"7b949c63-0745-43d1-a445-beefb836ad19\") "
Sep 15 22:54:18.297890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297511    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-tmp-dir\") pod \"3b155b8f-b07e-4801-984f-3609f6bbfc00\" (UID: \"3b155b8f-b07e-4801-984f-3609f6bbfc00\") "
Sep 15 22:54:18.297890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297590    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7b949c63-0745-43d1-a445-beefb836ad19" (UID: "7b949c63-0745-43d1-a445-beefb836ad19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.297890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297651    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-home" (OuterVolumeSpecName: "home") pod "7b949c63-0745-43d1-a445-beefb836ad19" (UID: "7b949c63-0745-43d1-a445-beefb836ad19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.297890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297680    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" (UID: "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.297890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297829    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-home" (OuterVolumeSpecName: "home") pod "3b155b8f-b07e-4801-984f-3609f6bbfc00" (UID: "3b155b8f-b07e-4801-984f-3609f6bbfc00"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.297890 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.297846    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b155b8f-b07e-4801-984f-3609f6bbfc00" (UID: "3b155b8f-b07e-4801-984f-3609f6bbfc00"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.298517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.298025    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" (UID: "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.298517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.298071    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.298517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.298090    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.298517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.298120    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.298517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.298133    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.298517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.298145    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.298517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.298158    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.298517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.298169    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.298517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.298226    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-model-cache" (OuterVolumeSpecName: "model-cache") pod "7b949c63-0745-43d1-a445-beefb836ad19" (UID: "7b949c63-0745-43d1-a445-beefb836ad19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.299897 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.299862    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b949c63-0745-43d1-a445-beefb836ad19-kube-api-access-sm9pg" (OuterVolumeSpecName: "kube-api-access-sm9pg") pod "7b949c63-0745-43d1-a445-beefb836ad19" (UID: "7b949c63-0745-43d1-a445-beefb836ad19"). InnerVolumeSpecName "kube-api-access-sm9pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:18.300493 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.300428    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" (UID: "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:18.300640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.300526    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b155b8f-b07e-4801-984f-3609f6bbfc00-kube-api-access-tslpl" (OuterVolumeSpecName: "kube-api-access-tslpl") pod "3b155b8f-b07e-4801-984f-3609f6bbfc00" (UID: "3b155b8f-b07e-4801-984f-3609f6bbfc00"). InnerVolumeSpecName "kube-api-access-tslpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:18.300640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.300520    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b155b8f-b07e-4801-984f-3609f6bbfc00-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b155b8f-b07e-4801-984f-3609f6bbfc00" (UID: "3b155b8f-b07e-4801-984f-3609f6bbfc00"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:18.300640 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.300596    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-kube-api-access-zh6l7" (OuterVolumeSpecName: "kube-api-access-zh6l7") pod "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" (UID: "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f"). InnerVolumeSpecName "kube-api-access-zh6l7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:18.300973 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.300951    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-dshm" (OuterVolumeSpecName: "dshm") pod "7b949c63-0745-43d1-a445-beefb836ad19" (UID: "7b949c63-0745-43d1-a445-beefb836ad19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.301475 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.301458    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b949c63-0745-43d1-a445-beefb836ad19-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7b949c63-0745-43d1-a445-beefb836ad19" (UID: "7b949c63-0745-43d1-a445-beefb836ad19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:18.301569 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.301546    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-dshm" (OuterVolumeSpecName: "dshm") pod "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" (UID: "0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.301913 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.301897    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-dshm" (OuterVolumeSpecName: "dshm") pod "3b155b8f-b07e-4801-984f-3609f6bbfc00" (UID: "3b155b8f-b07e-4801-984f-3609f6bbfc00"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:18.399501 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399406    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh6l7\" (UniqueName: \"kubernetes.io/projected/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-kube-api-access-zh6l7\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399501 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399440    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm9pg\" (UniqueName: \"kubernetes.io/projected/7b949c63-0745-43d1-a445-beefb836ad19-kube-api-access-sm9pg\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399501 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399451    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b155b8f-b07e-4801-984f-3609f6bbfc00-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399501 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399459    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399501 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399467    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tslpl\" (UniqueName: \"kubernetes.io/projected/3b155b8f-b07e-4801-984f-3609f6bbfc00-kube-api-access-tslpl\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399501 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399475    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b949c63-0745-43d1-a445-beefb836ad19-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399501 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399498    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399501 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399507    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399872 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399514    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b155b8f-b07e-4801-984f-3609f6bbfc00-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399872 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399521    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b949c63-0745-43d1-a445-beefb836ad19-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.399872 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.399530    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:18.607745 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.607711    2578 generic.go:356] "Generic (PLEG): container finished" podID="7b949c63-0745-43d1-a445-beefb836ad19" containerID="00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed" exitCode=0
Sep 15 22:54:18.607919 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.607756    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"
Sep 15 22:54:18.607919 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.607801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n" event={"ID":"7b949c63-0745-43d1-a445-beefb836ad19","Type":"ContainerDied","Data":"00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed"}
Sep 15 22:54:18.607919 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.607852    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n" event={"ID":"7b949c63-0745-43d1-a445-beefb836ad19","Type":"ContainerDied","Data":"8250aa4b02b62a0031c86e635682908c42d3c1007819389f576a557bdbd5d5fd"}
Sep 15 22:54:18.607919 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.607875    2578 scope.go:117] "RemoveContainer" containerID="00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed"
Sep 15 22:54:18.609227 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.609206    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b155b8f-b07e-4801-984f-3609f6bbfc00" containerID="ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c" exitCode=0
Sep 15 22:54:18.609337 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.609243    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"
Sep 15 22:54:18.609337 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.609299    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz" event={"ID":"3b155b8f-b07e-4801-984f-3609f6bbfc00","Type":"ContainerDied","Data":"ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c"}
Sep 15 22:54:18.609488 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.609342    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz" event={"ID":"3b155b8f-b07e-4801-984f-3609f6bbfc00","Type":"ContainerDied","Data":"5420165c867131a3695c64e276c7e4dc092804850c9eead2d56332fe6fefe84f"}
Sep 15 22:54:18.610844 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.610823    2578 generic.go:356] "Generic (PLEG): container finished" podID="0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" containerID="4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668" exitCode=0
Sep 15 22:54:18.610965 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.610866    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"
Sep 15 22:54:18.610965 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.610911    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb" event={"ID":"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f","Type":"ContainerDied","Data":"4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668"}
Sep 15 22:54:18.610965 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.610949    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb" event={"ID":"0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f","Type":"ContainerDied","Data":"be0dc3ae44634afbfb11aa1deb1672edd2193382b9118f673eaf9b89f56c3643"}
Sep 15 22:54:18.621220 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.621198    2578 scope.go:117] "RemoveContainer" containerID="00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed"
Sep 15 22:54:18.621557 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:18.621525    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed\": container with ID starting with 00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed not found: ID does not exist" containerID="00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed"
Sep 15 22:54:18.621659 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.621558    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed"} err="failed to get container status \"00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed\": rpc error: code = NotFound desc = could not find container \"00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed\": container with ID starting with 00c0ae7207518f7553d66b66d26077aef71c5a3564bd12c9c8837e0159931aed not found: ID does not exist"
Sep 15 22:54:18.621659 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.621582    2578 scope.go:117] "RemoveContainer" containerID="ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c"
Sep 15 22:54:18.629983 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.629966    2578 scope.go:117] "RemoveContainer" containerID="ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c"
Sep 15 22:54:18.630335 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:18.630315    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c\": container with ID starting with ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c not found: ID does not exist" containerID="ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c"
Sep 15 22:54:18.630398 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.630345    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c"} err="failed to get container status \"ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c\": rpc error: code = NotFound desc = could not find container \"ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c\": container with ID starting with ada069db2c13ce01187d31b256c0a39fc52a3692b9aed9bb3c3f96415d49530c not found: ID does not exist"
Sep 15 22:54:18.630398 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.630363    2578 scope.go:117] "RemoveContainer" containerID="4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668"
Sep 15 22:54:18.638997 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.638961    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"]
Sep 15 22:54:18.639961 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.639941    2578 scope.go:117] "RemoveContainer" containerID="4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668"
Sep 15 22:54:18.640361 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:18.640336    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668\": container with ID starting with 4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668 not found: ID does not exist" containerID="4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668"
Sep 15 22:54:18.640457 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.640367    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668"} err="failed to get container status \"4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668\": rpc error: code = NotFound desc = could not find container \"4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668\": container with ID starting with 4079f7f1e851de19493914fd32bfe8cbf6e28125e3ba77da1965ef9560a54668 not found: ID does not exist"
Sep 15 22:54:18.641882 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.641861    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/embedding-w0-fe020d-kserve-5976fc9449-qf52n"]
Sep 15 22:54:18.652981 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.652902    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"]
Sep 15 22:54:18.658861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.658834    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/simulator-w0-fe020d-kserve-8447fcb47b-mbfcz"]
Sep 15 22:54:18.669699 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.669666    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"]
Sep 15 22:54:18.671879 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.671854    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-fe020d/premium-w0-fe020d-kserve-7549c5488f-w9rcb"]
Sep 15 22:54:18.763581 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.763547    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" path="/var/lib/kubelet/pods/0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f/volumes"
Sep 15 22:54:18.763952 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.763939    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b155b8f-b07e-4801-984f-3609f6bbfc00" path="/var/lib/kubelet/pods/3b155b8f-b07e-4801-984f-3609f6bbfc00/volumes"
Sep 15 22:54:18.764347 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:18.764333    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b949c63-0745-43d1-a445-beefb836ad19" path="/var/lib/kubelet/pods/7b949c63-0745-43d1-a445-beefb836ad19/volumes"
Sep 15 22:54:20.447943 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.447919    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:54:20.523458 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523363    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-credential-socket\") pod \"058dbafa-8205-4b61-b96c-87d6a729374d\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") "
Sep 15 22:54:20.523458 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523416    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-istio-token\") pod \"058dbafa-8205-4b61-b96c-87d6a729374d\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") "
Sep 15 22:54:20.523458 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523450    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-envoy\") pod \"058dbafa-8205-4b61-b96c-87d6a729374d\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") "
Sep 15 22:54:20.523733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523479    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/058dbafa-8205-4b61-b96c-87d6a729374d-istiod-ca-cert\") pod \"058dbafa-8205-4b61-b96c-87d6a729374d\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") "
Sep 15 22:54:20.523733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523559    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-certs\") pod \"058dbafa-8205-4b61-b96c-87d6a729374d\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") "
Sep 15 22:54:20.523733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523589    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-data\") pod \"058dbafa-8205-4b61-b96c-87d6a729374d\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") "
Sep 15 22:54:20.523733 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523716    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "058dbafa-8205-4b61-b96c-87d6a729374d" (UID: "058dbafa-8205-4b61-b96c-87d6a729374d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:20.523937 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523736    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/058dbafa-8205-4b61-b96c-87d6a729374d-istio-podinfo\") pod \"058dbafa-8205-4b61-b96c-87d6a729374d\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") "
Sep 15 22:54:20.523937 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523783    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp2l5\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-kube-api-access-dp2l5\") pod \"058dbafa-8205-4b61-b96c-87d6a729374d\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") "
Sep 15 22:54:20.523937 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523861    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-socket\") pod \"058dbafa-8205-4b61-b96c-87d6a729374d\" (UID: \"058dbafa-8205-4b61-b96c-87d6a729374d\") "
Sep 15 22:54:20.523937 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523869    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-data" (OuterVolumeSpecName: "istio-data") pod "058dbafa-8205-4b61-b96c-87d6a729374d" (UID: "058dbafa-8205-4b61-b96c-87d6a729374d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:20.523937 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.523889    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "058dbafa-8205-4b61-b96c-87d6a729374d" (UID: "058dbafa-8205-4b61-b96c-87d6a729374d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:20.524228 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.524022    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/058dbafa-8205-4b61-b96c-87d6a729374d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "058dbafa-8205-4b61-b96c-87d6a729374d" (UID: "058dbafa-8205-4b61-b96c-87d6a729374d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:20.524228 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.524211    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "058dbafa-8205-4b61-b96c-87d6a729374d" (UID: "058dbafa-8205-4b61-b96c-87d6a729374d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:20.524416 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.524396    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/058dbafa-8205-4b61-b96c-87d6a729374d-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:20.524459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.524424    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:20.524459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.524438    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:20.524459 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.524451    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:20.524567 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.524465    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:20.525760 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.525735    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "058dbafa-8205-4b61-b96c-87d6a729374d" (UID: "058dbafa-8205-4b61-b96c-87d6a729374d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:20.525760 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.525737    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-istio-token" (OuterVolumeSpecName: "istio-token") pod "058dbafa-8205-4b61-b96c-87d6a729374d" (UID: "058dbafa-8205-4b61-b96c-87d6a729374d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:20.526158 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.526138    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-kube-api-access-dp2l5" (OuterVolumeSpecName: "kube-api-access-dp2l5") pod "058dbafa-8205-4b61-b96c-87d6a729374d" (UID: "058dbafa-8205-4b61-b96c-87d6a729374d"). InnerVolumeSpecName "kube-api-access-dp2l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:20.526227 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.526141    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/058dbafa-8205-4b61-b96c-87d6a729374d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "058dbafa-8205-4b61-b96c-87d6a729374d" (UID: "058dbafa-8205-4b61-b96c-87d6a729374d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:54:20.625046 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.625013    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:20.625046 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.625043    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/058dbafa-8205-4b61-b96c-87d6a729374d-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:20.625296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.625057    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/058dbafa-8205-4b61-b96c-87d6a729374d-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:20.625296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.625070    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp2l5\" (UniqueName: \"kubernetes.io/projected/058dbafa-8205-4b61-b96c-87d6a729374d-kube-api-access-dp2l5\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:20.625296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.625148    2578 generic.go:356] "Generic (PLEG): container finished" podID="058dbafa-8205-4b61-b96c-87d6a729374d" containerID="7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83" exitCode=0
Sep 15 22:54:20.625296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.625214    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"
Sep 15 22:54:20.625296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.625229    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" event={"ID":"058dbafa-8205-4b61-b96c-87d6a729374d","Type":"ContainerDied","Data":"7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83"}
Sep 15 22:54:20.625296 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.625285    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f" event={"ID":"058dbafa-8205-4b61-b96c-87d6a729374d","Type":"ContainerDied","Data":"d38b2b00914a9819a344f548dc03e61f932de61322e61213db6d3d641a6eda80"}
Sep 15 22:54:20.625563 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.625307    2578 scope.go:117] "RemoveContainer" containerID="7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83"
Sep 15 22:54:20.634546 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.634524    2578 scope.go:117] "RemoveContainer" containerID="50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9"
Sep 15 22:54:20.642693 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.642674    2578 scope.go:117] "RemoveContainer" containerID="7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83"
Sep 15 22:54:20.642953 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:20.642935    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83\": container with ID starting with 7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83 not found: ID does not exist" containerID="7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83"
Sep 15 22:54:20.643020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.642961    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83"} err="failed to get container status \"7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83\": rpc error: code = NotFound desc = could not find container \"7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83\": container with ID starting with 7186f01477e9313793ecade272cfc6caf6669dc89d37deedc985a0da54a6cd83 not found: ID does not exist"
Sep 15 22:54:20.643020 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.642979    2578 scope.go:117] "RemoveContainer" containerID="50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9"
Sep 15 22:54:20.643246 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:20.643233    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9\": container with ID starting with 50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9 not found: ID does not exist" containerID="50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9"
Sep 15 22:54:20.643290 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.643250    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9"} err="failed to get container status \"50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9\": rpc error: code = NotFound desc = could not find container \"50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9\": container with ID starting with 50d68058beb7022235c46ff65dae083b615c677606595524499c0f0e1cb4d2d9 not found: ID does not exist"
Sep 15 22:54:20.647350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.647328    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"]
Sep 15 22:54:20.651608 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.651584    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f2241f-openshift-default-6855b78d84-gtd4f"]
Sep 15 22:54:20.763548 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:20.763507    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" path="/var/lib/kubelet/pods/058dbafa-8205-4b61-b96c-87d6a729374d/volumes"
Sep 15 22:54:32.818047 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:32.817951    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"]
Sep 15 22:54:32.818638 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:32.818302    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy" containerID="cri-o://69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d" gracePeriod=30
Sep 15 22:54:38.044517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.044478    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"]
Sep 15 22:54:38.077611 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.077576    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:54:38.191414 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191373    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q62lv\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-kube-api-access-q62lv\") pod \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") "
Sep 15 22:54:38.191414 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191425    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-envoy\") pod \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") "
Sep 15 22:54:38.191676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191446    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-token\") pod \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") "
Sep 15 22:54:38.191676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191477    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-data\") pod \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") "
Sep 15 22:54:38.191676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191500    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-credential-socket\") pod \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") "
Sep 15 22:54:38.191676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191518    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-socket\") pod \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") "
Sep 15 22:54:38.191676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191560    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istiod-ca-cert\") pod \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") "
Sep 15 22:54:38.191676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191587    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-podinfo\") pod \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") "
Sep 15 22:54:38.191676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191628    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-certs\") pod \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\" (UID: \"1b84e575-ed0f-42a3-a8b1-e4122a09997e\") "
Sep 15 22:54:38.192033 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191880    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b84e575-ed0f-42a3-a8b1-e4122a09997e" (UID: "1b84e575-ed0f-42a3-a8b1-e4122a09997e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:38.192033 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191890    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b84e575-ed0f-42a3-a8b1-e4122a09997e" (UID: "1b84e575-ed0f-42a3-a8b1-e4122a09997e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:38.192033 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.191904    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b84e575-ed0f-42a3-a8b1-e4122a09997e" (UID: "1b84e575-ed0f-42a3-a8b1-e4122a09997e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:38.192033 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.192015    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:38.192284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.192062    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:38.192284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.192075    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:38.192284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.192143    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b84e575-ed0f-42a3-a8b1-e4122a09997e" (UID: "1b84e575-ed0f-42a3-a8b1-e4122a09997e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:38.192284 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.192208    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b84e575-ed0f-42a3-a8b1-e4122a09997e" (UID: "1b84e575-ed0f-42a3-a8b1-e4122a09997e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:38.193838 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.193809    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b84e575-ed0f-42a3-a8b1-e4122a09997e" (UID: "1b84e575-ed0f-42a3-a8b1-e4122a09997e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:38.193949 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.193895    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b84e575-ed0f-42a3-a8b1-e4122a09997e" (UID: "1b84e575-ed0f-42a3-a8b1-e4122a09997e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:54:38.194133 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.194087    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b84e575-ed0f-42a3-a8b1-e4122a09997e" (UID: "1b84e575-ed0f-42a3-a8b1-e4122a09997e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:38.194133 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.194106    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-kube-api-access-q62lv" (OuterVolumeSpecName: "kube-api-access-q62lv") pod "1b84e575-ed0f-42a3-a8b1-e4122a09997e" (UID: "1b84e575-ed0f-42a3-a8b1-e4122a09997e"). InnerVolumeSpecName "kube-api-access-q62lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:38.293204 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.293165    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:38.293204 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.293195    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:38.293204 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.293204    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:38.293204 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.293213    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q62lv\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-kube-api-access-q62lv\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:38.293477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.293224    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:38.293477 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.293232    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b84e575-ed0f-42a3-a8b1-e4122a09997e-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:38.716190 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.716153    2578 generic.go:356] "Generic (PLEG): container finished" podID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerID="69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d" exitCode=0
Sep 15 22:54:38.716378 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.716206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" event={"ID":"1b84e575-ed0f-42a3-a8b1-e4122a09997e","Type":"ContainerDied","Data":"69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d"}
Sep 15 22:54:38.716378 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.716214    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"
Sep 15 22:54:38.716378 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.716257    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c" event={"ID":"1b84e575-ed0f-42a3-a8b1-e4122a09997e","Type":"ContainerDied","Data":"a3ed126361e105f81247d32e0c06595d351d361a5ab664900d23d27d15848876"}
Sep 15 22:54:38.716378 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.716282    2578 scope.go:117] "RemoveContainer" containerID="69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d"
Sep 15 22:54:38.731711 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.731690    2578 scope.go:117] "RemoveContainer" containerID="58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403"
Sep 15 22:54:38.740686 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.740665    2578 scope.go:117] "RemoveContainer" containerID="69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d"
Sep 15 22:54:38.741048 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:38.741026    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d\": container with ID starting with 69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d not found: ID does not exist" containerID="69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d"
Sep 15 22:54:38.741159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.741057    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d"} err="failed to get container status \"69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d\": rpc error: code = NotFound desc = could not find container \"69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d\": container with ID starting with 69d3a013a4ffc8ed261fe18b29e615e3b59e4f975025a4ad95edd3141233c46d not found: ID does not exist"
Sep 15 22:54:38.741159 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.741076    2578 scope.go:117] "RemoveContainer" containerID="58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403"
Sep 15 22:54:38.741381 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:38.741366    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403\": container with ID starting with 58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403 not found: ID does not exist" containerID="58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403"
Sep 15 22:54:38.741422 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.741393    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403"} err="failed to get container status \"58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403\": rpc error: code = NotFound desc = could not find container \"58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403\": container with ID starting with 58b3e2eb1fb3e70f751a67ad41f13260535bebef85b4ab2816b94070ec123403 not found: ID does not exist"
Sep 15 22:54:38.744755 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.744727    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"]
Sep 15 22:54:38.748161 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.748135    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4f70a9-openshift-default-57fb65bbc4-gkb8c"]
Sep 15 22:54:38.763414 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:38.763388    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" path="/var/lib/kubelet/pods/1b84e575-ed0f-42a3-a8b1-e4122a09997e/volumes"
Sep 15 22:54:53.064352 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.064282    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979" podUID="8b9ef65a-1d7f-438b-9503-cfb8b0963503" containerName="main" containerID="cri-o://4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c" gracePeriod=45
Sep 15 22:54:53.318250 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.318184    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:54:53.439401 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.439369    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wjzs\" (UniqueName: \"kubernetes.io/projected/8b9ef65a-1d7f-438b-9503-cfb8b0963503-kube-api-access-5wjzs\") pod \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") "
Sep 15 22:54:53.439601 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.439420    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tmp-dir\") pod \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") "
Sep 15 22:54:53.439601 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.439441    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-dshm\") pod \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") "
Sep 15 22:54:53.439601 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.439467    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-home\") pod \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") "
Sep 15 22:54:53.439601 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.439523    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tls-certs\") pod \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") "
Sep 15 22:54:53.439824 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.439714    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b9ef65a-1d7f-438b-9503-cfb8b0963503" (UID: "8b9ef65a-1d7f-438b-9503-cfb8b0963503"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:53.439824 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.439780    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-home" (OuterVolumeSpecName: "home") pod "8b9ef65a-1d7f-438b-9503-cfb8b0963503" (UID: "8b9ef65a-1d7f-438b-9503-cfb8b0963503"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:53.439961 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.439944    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-model-cache\") pod \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\" (UID: \"8b9ef65a-1d7f-438b-9503-cfb8b0963503\") "
Sep 15 22:54:53.440057 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.440017    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b9ef65a-1d7f-438b-9503-cfb8b0963503" (UID: "8b9ef65a-1d7f-438b-9503-cfb8b0963503"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:53.440395 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.440376    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:53.440476 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.440400    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:53.440476 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.440413    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:53.441708 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.441681    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9ef65a-1d7f-438b-9503-cfb8b0963503-kube-api-access-5wjzs" (OuterVolumeSpecName: "kube-api-access-5wjzs") pod "8b9ef65a-1d7f-438b-9503-cfb8b0963503" (UID: "8b9ef65a-1d7f-438b-9503-cfb8b0963503"). InnerVolumeSpecName "kube-api-access-5wjzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:53.441800 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.441719    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-dshm" (OuterVolumeSpecName: "dshm") pod "8b9ef65a-1d7f-438b-9503-cfb8b0963503" (UID: "8b9ef65a-1d7f-438b-9503-cfb8b0963503"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:54:53.441988 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.441967    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b9ef65a-1d7f-438b-9503-cfb8b0963503" (UID: "8b9ef65a-1d7f-438b-9503-cfb8b0963503"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:53.541390 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.541357    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wjzs\" (UniqueName: \"kubernetes.io/projected/8b9ef65a-1d7f-438b-9503-cfb8b0963503-kube-api-access-5wjzs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:53.541390 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.541385    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9ef65a-1d7f-438b-9503-cfb8b0963503-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:53.541390 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.541398    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9ef65a-1d7f-438b-9503-cfb8b0963503-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:54:53.791861 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.791828    2578 generic.go:356] "Generic (PLEG): container finished" podID="8b9ef65a-1d7f-438b-9503-cfb8b0963503" containerID="4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c" exitCode=0
Sep 15 22:54:53.792068 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.791875    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"
Sep 15 22:54:53.792068 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.791887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979" event={"ID":"8b9ef65a-1d7f-438b-9503-cfb8b0963503","Type":"ContainerDied","Data":"4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c"}
Sep 15 22:54:53.792068 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.791939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979" event={"ID":"8b9ef65a-1d7f-438b-9503-cfb8b0963503","Type":"ContainerDied","Data":"54482be5753f76e8cea2617dd206c81ba2568587ce47eedcb75fa454e07930e0"}
Sep 15 22:54:53.792068 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.791961    2578 scope.go:117] "RemoveContainer" containerID="4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c"
Sep 15 22:54:53.801001 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.800978    2578 scope.go:117] "RemoveContainer" containerID="4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c"
Sep 15 22:54:53.801322 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:54:53.801300    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c\": container with ID starting with 4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c not found: ID does not exist" containerID="4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c"
Sep 15 22:54:53.801400 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.801330    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c"} err="failed to get container status \"4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c\": rpc error: code = NotFound desc = could not find container \"4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c\": container with ID starting with 4f90750400919eb8ef1d087434108912b19814ea8ea1f10109b35e3502df897c not found: ID does not exist"
Sep 15 22:54:53.813704 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.813672    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"]
Sep 15 22:54:53.816348 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:53.816323    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-03f258/simulator-w4-03f258-kserve-69856cb965-lh979"]
Sep 15 22:54:54.763417 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:54.763381    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b9ef65a-1d7f-438b-9503-cfb8b0963503" path="/var/lib/kubelet/pods/8b9ef65a-1d7f-438b-9503-cfb8b0963503/volumes"
Sep 15 22:54:56.167131 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:56.167084    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"]
Sep 15 22:54:56.167571 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:54:56.167326    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy" containerID="cri-o://d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be" gracePeriod=30
Sep 15 22:55:01.419533 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.419506    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:55:01.522450 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522418    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-envoy\") pod \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") "
Sep 15 22:55:01.522663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522470    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-credential-socket\") pod \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") "
Sep 15 22:55:01.522663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522498    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-socket\") pod \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") "
Sep 15 22:55:01.522663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522540    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cgg2\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-kube-api-access-7cgg2\") pod \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") "
Sep 15 22:55:01.522663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522557    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-token\") pod \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") "
Sep 15 22:55:01.522663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522576    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-certs\") pod \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") "
Sep 15 22:55:01.522663 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522622    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-data\") pod \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") "
Sep 15 22:55:01.522960 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522724    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istiod-ca-cert\") pod \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") "
Sep 15 22:55:01.522960 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522781    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-podinfo\") pod \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\" (UID: \"9e1a1aa4-d1ce-49c3-a406-025acfe74568\") "
Sep 15 22:55:01.522960 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522909    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e1a1aa4-d1ce-49c3-a406-025acfe74568" (UID: "9e1a1aa4-d1ce-49c3-a406-025acfe74568"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:55:01.522960 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522924    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e1a1aa4-d1ce-49c3-a406-025acfe74568" (UID: "9e1a1aa4-d1ce-49c3-a406-025acfe74568"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:55:01.522960 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522940    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e1a1aa4-d1ce-49c3-a406-025acfe74568" (UID: "9e1a1aa4-d1ce-49c3-a406-025acfe74568"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:55:01.522960 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.522949    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e1a1aa4-d1ce-49c3-a406-025acfe74568" (UID: "9e1a1aa4-d1ce-49c3-a406-025acfe74568"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:55:01.523283 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.523213    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:01.523283 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.523234    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:01.523283 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.523251    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:01.523283 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.523264    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:01.523436 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.523279    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e1a1aa4-d1ce-49c3-a406-025acfe74568" (UID: "9e1a1aa4-d1ce-49c3-a406-025acfe74568"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:55:01.524989 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.524966    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e1a1aa4-d1ce-49c3-a406-025acfe74568" (UID: "9e1a1aa4-d1ce-49c3-a406-025acfe74568"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:55:01.525138 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.525016    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e1a1aa4-d1ce-49c3-a406-025acfe74568" (UID: "9e1a1aa4-d1ce-49c3-a406-025acfe74568"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 22:55:01.525369 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.525347    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-kube-api-access-7cgg2" (OuterVolumeSpecName: "kube-api-access-7cgg2") pod "9e1a1aa4-d1ce-49c3-a406-025acfe74568" (UID: "9e1a1aa4-d1ce-49c3-a406-025acfe74568"). InnerVolumeSpecName "kube-api-access-7cgg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:55:01.525435 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.525349    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e1a1aa4-d1ce-49c3-a406-025acfe74568" (UID: "9e1a1aa4-d1ce-49c3-a406-025acfe74568"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:55:01.624517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.624483    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cgg2\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-kube-api-access-7cgg2\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:01.624517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.624514    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:01.624517 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.624526    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:01.624771 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.624534    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:01.624771 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.624542    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e1a1aa4-d1ce-49c3-a406-025acfe74568-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:01.832893 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.832800    2578 generic.go:356] "Generic (PLEG): container finished" podID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerID="d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be" exitCode=0
Sep 15 22:55:01.832893 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.832846    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"
Sep 15 22:55:01.833147 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.832849    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" event={"ID":"9e1a1aa4-d1ce-49c3-a406-025acfe74568","Type":"ContainerDied","Data":"d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be"}
Sep 15 22:55:01.833147 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.832962    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65" event={"ID":"9e1a1aa4-d1ce-49c3-a406-025acfe74568","Type":"ContainerDied","Data":"6ebac159987db9ee9fc556905e06ca42790067059922d9e025f5a52cac1d0adb"}
Sep 15 22:55:01.833147 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.832992    2578 scope.go:117] "RemoveContainer" containerID="d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be"
Sep 15 22:55:01.842230 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.842212    2578 scope.go:117] "RemoveContainer" containerID="380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3"
Sep 15 22:55:01.850053 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.850035    2578 scope.go:117] "RemoveContainer" containerID="d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be"
Sep 15 22:55:01.850353 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:55:01.850333    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be\": container with ID starting with d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be not found: ID does not exist" containerID="d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be"
Sep 15 22:55:01.850428 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.850360    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be"} err="failed to get container status \"d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be\": rpc error: code = NotFound desc = could not find container \"d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be\": container with ID starting with d57e14ba67afcfaf585ca299123ab8e773f014273222c7417b3f6a191d6bd5be not found: ID does not exist"
Sep 15 22:55:01.850428 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.850381    2578 scope.go:117] "RemoveContainer" containerID="380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3"
Sep 15 22:55:01.850629 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:55:01.850613    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3\": container with ID starting with 380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3 not found: ID does not exist" containerID="380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3"
Sep 15 22:55:01.850676 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.850635    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3"} err="failed to get container status \"380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3\": rpc error: code = NotFound desc = could not find container \"380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3\": container with ID starting with 380c18573137bd8e4736966d9bc7927983bfbda1b4c562e3ba0edb313157b7e3 not found: ID does not exist"
Sep 15 22:55:01.858800 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.858770    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"]
Sep 15 22:55:01.862758 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:01.862729    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-03f258-openshift-default-6c5c7cdccb-8ch65"]
Sep 15 22:55:02.763530 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:02.763496    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" path="/var/lib/kubelet/pods/9e1a1aa4-d1ce-49c3-a406-025acfe74568/volumes"
Sep 15 22:55:42.249490 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:42.249456    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"]
Sep 15 22:55:57.268961 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.268917    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d" podUID="50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" containerName="main" containerID="cri-o://22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc" gracePeriod=45
Sep 15 22:55:57.539033 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.539007    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:55:57.638983 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.638948    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-dshm\") pod \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") "
Sep 15 22:55:57.638983 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.638990    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tls-certs\") pod \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") "
Sep 15 22:55:57.639242 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639057    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tmp-dir\") pod \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") "
Sep 15 22:55:57.639242 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639087    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbtjq\" (UniqueName: \"kubernetes.io/projected/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-kube-api-access-dbtjq\") pod \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") "
Sep 15 22:55:57.639242 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639149    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-home\") pod \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") "
Sep 15 22:55:57.639242 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639213    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-model-cache\") pod \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\" (UID: \"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b\") "
Sep 15 22:55:57.639464 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639376    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" (UID: "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:55:57.639527 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639469    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-home" (OuterVolumeSpecName: "home") pod "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" (UID: "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:55:57.639585 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639529    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-model-cache" (OuterVolumeSpecName: "model-cache") pod "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" (UID: "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:55:57.639636 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639615    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:57.639636 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639632    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:57.639719 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.639641    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:57.641359 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.641337    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" (UID: "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:55:57.641634 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.641609    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-dshm" (OuterVolumeSpecName: "dshm") pod "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" (UID: "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:55:57.641730 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.641613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-kube-api-access-dbtjq" (OuterVolumeSpecName: "kube-api-access-dbtjq") pod "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" (UID: "50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b"). InnerVolumeSpecName "kube-api-access-dbtjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:55:57.740849 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.740810    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:57.740849 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.740843    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:57.740849 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:57.740854    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbtjq\" (UniqueName: \"kubernetes.io/projected/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b-kube-api-access-dbtjq\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:55:58.104500 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.104461    2578 generic.go:356] "Generic (PLEG): container finished" podID="50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" containerID="22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc" exitCode=0
Sep 15 22:55:58.104500 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.104500    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"
Sep 15 22:55:58.104743 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.104539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d" event={"ID":"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b","Type":"ContainerDied","Data":"22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc"}
Sep 15 22:55:58.104743 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.104575    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d" event={"ID":"50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b","Type":"ContainerDied","Data":"c232908c92e4a3bd46a8e17a05b656e8b4c2a7eb53e5e536b64bc1409e3e188a"}
Sep 15 22:55:58.104743 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.104595    2578 scope.go:117] "RemoveContainer" containerID="22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc"
Sep 15 22:55:58.113383 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.113360    2578 scope.go:117] "RemoveContainer" containerID="22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc"
Sep 15 22:55:58.113718 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:55:58.113698    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc\": container with ID starting with 22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc not found: ID does not exist" containerID="22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc"
Sep 15 22:55:58.113796 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.113728    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc"} err="failed to get container status \"22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc\": rpc error: code = NotFound desc = could not find container \"22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc\": container with ID starting with 22e520ad70896a63346efc6c8a5ee56d61063a21c78f949b84a4a2490259d5dc not found: ID does not exist"
Sep 15 22:55:58.126203 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.126173    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"]
Sep 15 22:55:58.129808 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.129785    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d"]
Sep 15 22:55:58.767626 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:55:58.767590    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" path="/var/lib/kubelet/pods/50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b/volumes"
Sep 15 22:56:02.527965 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:02.527855    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w3-0016d4/auth-test-model-0016d4-kserve-596496bdcf-6sl8d" podUID="50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 15 22:56:58.708534 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708498    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7654-547a0d"]
Sep 15 22:56:58.708958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708889    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" containerName="main"
Sep 15 22:56:58.708958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708926    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" containerName="main"
Sep 15 22:56:58.708958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708936    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy"
Sep 15 22:56:58.708958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708943    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy"
Sep 15 22:56:58.708958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708951    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy"
Sep 15 22:56:58.708958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708956    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708964    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56267239-c3a7-4494-b01b-c0c73b9fcb1f" containerName="maas-api"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708969    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="56267239-c3a7-4494-b01b-c0c73b9fcb1f" containerName="maas-api"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708976    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708981    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.708999    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709004    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709009    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b155b8f-b07e-4801-984f-3609f6bbfc00" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709014    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b155b8f-b07e-4801-984f-3609f6bbfc00" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709021    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709026    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709034    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709044    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709051    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709056    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709062    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709067    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709072    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9ef65a-1d7f-438b-9503-cfb8b0963503" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709077    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9ef65a-1d7f-438b-9503-cfb8b0963503" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709083    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="963724ba-d871-4de3-8ed9-958b31fd6c63" containerName="maas-api"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709087    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="963724ba-d871-4de3-8ed9-958b31fd6c63" containerName="maas-api"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709107    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709112    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709120    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b949c63-0745-43d1-a445-beefb836ad19" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709125    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b949c63-0745-43d1-a445-beefb836ad19" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709186    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f22a4a-bab3-4c6f-beb4-c2a4f7246d6b" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709195    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ff99328-e4a2-4eb8-aa27-bd78fa34bc5f" containerName="main"
Sep 15 22:56:58.709191 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709202    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709209    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709214    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9ef65a-1d7f-438b-9503-cfb8b0963503" containerName="main"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709219    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b155b8f-b07e-4801-984f-3609f6bbfc00" containerName="main"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709223    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b949c63-0745-43d1-a445-beefb836ad19" containerName="main"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709227    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709233    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709238    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="058dbafa-8205-4b61-b96c-87d6a729374d" containerName="istio-proxy"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709242    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b84e575-ed0f-42a3-a8b1-e4122a09997e" containerName="istio-proxy"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709247    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="963724ba-d871-4de3-8ed9-958b31fd6c63" containerName="maas-api"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709252    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1a1aa4-d1ce-49c3-a406-025acfe74568" containerName="istio-proxy"
Sep 15 22:56:58.710212 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.709256    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="56267239-c3a7-4494-b01b-c0c73b9fcb1f" containerName="maas-api"
Sep 15 22:56:58.712254 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.712237    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-547a0d"
Sep 15 22:56:58.714879 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.714858    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4fdg9\""
Sep 15 22:56:58.720125 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.720085    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-547a0d"]
Sep 15 22:56:58.802978 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.802842    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgqw9\" (UniqueName: \"kubernetes.io/projected/45c7d30f-daaa-4834-ac9e-94d51e9aa5c3-kube-api-access-mgqw9\") pod \"test-curl-7654-547a0d\" (UID: \"45c7d30f-daaa-4834-ac9e-94d51e9aa5c3\") " pod="opendatahub/test-curl-7654-547a0d"
Sep 15 22:56:58.903765 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.903716    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgqw9\" (UniqueName: \"kubernetes.io/projected/45c7d30f-daaa-4834-ac9e-94d51e9aa5c3-kube-api-access-mgqw9\") pod \"test-curl-7654-547a0d\" (UID: \"45c7d30f-daaa-4834-ac9e-94d51e9aa5c3\") " pod="opendatahub/test-curl-7654-547a0d"
Sep 15 22:56:58.913617 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:58.913581    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgqw9\" (UniqueName: \"kubernetes.io/projected/45c7d30f-daaa-4834-ac9e-94d51e9aa5c3-kube-api-access-mgqw9\") pod \"test-curl-7654-547a0d\" (UID: \"45c7d30f-daaa-4834-ac9e-94d51e9aa5c3\") " pod="opendatahub/test-curl-7654-547a0d"
Sep 15 22:56:59.021662 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:59.021567    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-547a0d"
Sep 15 22:56:59.291732 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:59.291698    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-547a0d"]
Sep 15 22:56:59.294556 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:56:59.294527    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45c7d30f_daaa_4834_ac9e_94d51e9aa5c3.slice/crio-bd46ba4a53558ad6852bef44986c34a16da5514b883021e0f51b4f3a5ff425f3 WatchSource:0}: Error finding container bd46ba4a53558ad6852bef44986c34a16da5514b883021e0f51b4f3a5ff425f3: Status 404 returned error can't find the container with id bd46ba4a53558ad6852bef44986c34a16da5514b883021e0f51b4f3a5ff425f3
Sep 15 22:56:59.386620 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:59.386578    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-547a0d" event={"ID":"45c7d30f-daaa-4834-ac9e-94d51e9aa5c3","Type":"ContainerStarted","Data":"5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67"}
Sep 15 22:56:59.386789 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:59.386629    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-547a0d" event={"ID":"45c7d30f-daaa-4834-ac9e-94d51e9aa5c3","Type":"ContainerStarted","Data":"bd46ba4a53558ad6852bef44986c34a16da5514b883021e0f51b4f3a5ff425f3"}
Sep 15 22:56:59.400492 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:59.400443    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7654-547a0d" podStartSLOduration=1.400426222 podStartE2EDuration="1.400426222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:56:59.400203142 +0000 UTC m=+1759.153827272" watchObservedRunningTime="2026-09-15 22:56:59.400426222 +0000 UTC m=+1759.154050360"
Sep 15 22:56:59.982211 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:59.982177    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7654-547a0d"]
Sep 15 22:56:59.986148 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:56:59.986119    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7654-547a0d"]
Sep 15 22:57:00.096175 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.096138    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7654-6e9026"]
Sep 15 22:57:00.096531 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.096518    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45c7d30f-daaa-4834-ac9e-94d51e9aa5c3" containerName="test-curl-7654-547a0d"
Sep 15 22:57:00.096575 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.096532    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="45c7d30f-daaa-4834-ac9e-94d51e9aa5c3" containerName="test-curl-7654-547a0d"
Sep 15 22:57:00.096629 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.096613    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="241305ff-629f-4468-a196-c7b57b0a9470" containerName="istio-proxy"
Sep 15 22:57:00.096667 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.096635    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="45c7d30f-daaa-4834-ac9e-94d51e9aa5c3" containerName="test-curl-7654-547a0d"
Sep 15 22:57:00.099492 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.099475    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-6e9026"
Sep 15 22:57:00.106341 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.106313    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-6e9026"]
Sep 15 22:57:00.215917 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.215880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6vsn\" (UniqueName: \"kubernetes.io/projected/2d387e4a-fe84-4a3d-aeee-f8b861944631-kube-api-access-s6vsn\") pod \"test-curl-7654-6e9026\" (UID: \"2d387e4a-fe84-4a3d-aeee-f8b861944631\") " pod="opendatahub/test-curl-7654-6e9026"
Sep 15 22:57:00.317506 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.317405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6vsn\" (UniqueName: \"kubernetes.io/projected/2d387e4a-fe84-4a3d-aeee-f8b861944631-kube-api-access-s6vsn\") pod \"test-curl-7654-6e9026\" (UID: \"2d387e4a-fe84-4a3d-aeee-f8b861944631\") " pod="opendatahub/test-curl-7654-6e9026"
Sep 15 22:57:00.326138 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.326113    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6vsn\" (UniqueName: \"kubernetes.io/projected/2d387e4a-fe84-4a3d-aeee-f8b861944631-kube-api-access-s6vsn\") pod \"test-curl-7654-6e9026\" (UID: \"2d387e4a-fe84-4a3d-aeee-f8b861944631\") " pod="opendatahub/test-curl-7654-6e9026"
Sep 15 22:57:00.391192 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:57:00.391160    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7654-547a0d" container="test-curl-7654-547a0d"
Sep 15 22:57:00.409881 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.409849    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-6e9026"
Sep 15 22:57:00.678163 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:00.678115    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-6e9026"]
Sep 15 22:57:00.680823 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:57:00.680791    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d387e4a_fe84_4a3d_aeee_f8b861944631.slice/crio-8a5cf84f75f7264fe311de3e16bf5ede95b21f13b6faea4b253a22ba35ca2c17 WatchSource:0}: Error finding container 8a5cf84f75f7264fe311de3e16bf5ede95b21f13b6faea4b253a22ba35ca2c17: Status 404 returned error can't find the container with id 8a5cf84f75f7264fe311de3e16bf5ede95b21f13b6faea4b253a22ba35ca2c17
Sep 15 22:57:01.397235 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:01.397193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-6e9026" event={"ID":"2d387e4a-fe84-4a3d-aeee-f8b861944631","Type":"ContainerStarted","Data":"4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a"}
Sep 15 22:57:01.397681 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:01.397246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-6e9026" event={"ID":"2d387e4a-fe84-4a3d-aeee-f8b861944631","Type":"ContainerStarted","Data":"8a5cf84f75f7264fe311de3e16bf5ede95b21f13b6faea4b253a22ba35ca2c17"}
Sep 15 22:57:01.397681 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:01.397289    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7654-547a0d" podUID="45c7d30f-daaa-4834-ac9e-94d51e9aa5c3" containerName="test-curl-7654-547a0d" containerID="cri-o://5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67" gracePeriod=2
Sep 15 22:57:01.418792 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:01.418727    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7654-6e9026" podStartSLOduration=1.418705618 podStartE2EDuration="1.418705618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:57:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:57:01.41574312 +0000 UTC m=+1761.169367248" watchObservedRunningTime="2026-09-15 22:57:01.418705618 +0000 UTC m=+1761.172329747"
Sep 15 22:57:01.979077 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:01.978978    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7654-6e9026"]
Sep 15 22:57:01.984421 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:01.984383    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7654-6e9026"]
Sep 15 22:57:02.402436 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:57:02.402400    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7654-6e9026" container="test-curl-7654-6e9026"
Sep 15 22:57:03.408033 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:03.407993    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7654-6e9026" podUID="2d387e4a-fe84-4a3d-aeee-f8b861944631" containerName="test-curl-7654-6e9026" containerID="cri-o://4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a" gracePeriod=2
Sep 15 22:57:03.566741 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:03.566713    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-547a0d"
Sep 15 22:57:03.651974 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:03.651876    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgqw9\" (UniqueName: \"kubernetes.io/projected/45c7d30f-daaa-4834-ac9e-94d51e9aa5c3-kube-api-access-mgqw9\") pod \"45c7d30f-daaa-4834-ac9e-94d51e9aa5c3\" (UID: \"45c7d30f-daaa-4834-ac9e-94d51e9aa5c3\") "
Sep 15 22:57:03.654048 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:03.654017    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45c7d30f-daaa-4834-ac9e-94d51e9aa5c3-kube-api-access-mgqw9" (OuterVolumeSpecName: "kube-api-access-mgqw9") pod "45c7d30f-daaa-4834-ac9e-94d51e9aa5c3" (UID: "45c7d30f-daaa-4834-ac9e-94d51e9aa5c3"). InnerVolumeSpecName "kube-api-access-mgqw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:57:03.753002 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:03.752962    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgqw9\" (UniqueName: \"kubernetes.io/projected/45c7d30f-daaa-4834-ac9e-94d51e9aa5c3-kube-api-access-mgqw9\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:57:04.413483 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:04.413449    2578 generic.go:356] "Generic (PLEG): container finished" podID="45c7d30f-daaa-4834-ac9e-94d51e9aa5c3" containerID="5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67" exitCode=137
Sep 15 22:57:04.414054 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:04.413493    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-547a0d"
Sep 15 22:57:04.414054 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:04.413510    2578 scope.go:117] "RemoveContainer" containerID="5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67"
Sep 15 22:57:04.423958 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:04.423931    2578 scope.go:117] "RemoveContainer" containerID="5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67"
Sep 15 22:57:04.424791 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:57:04.424754    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67\": container with ID starting with 5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67 not found: ID does not exist" containerID="5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67"
Sep 15 22:57:04.424894 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:04.424794    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67"} err="failed to get container status \"5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67\": rpc error: code = NotFound desc = could not find container \"5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67\": container with ID starting with 5e2bc0046a93af1d505d2044998abddd5c6d709159d8f2cd45dabb2af2ee5a67 not found: ID does not exist"
Sep 15 22:57:04.427008 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:57:04.426987    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7654-547a0d\" is forbidden: User \"system:node:ip-10-0-128-247.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-247.ec2.internal' and this object" podUID="45c7d30f-daaa-4834-ac9e-94d51e9aa5c3" pod="opendatahub/test-curl-7654-547a0d"
Sep 15 22:57:04.763180 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:04.763147    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45c7d30f-daaa-4834-ac9e-94d51e9aa5c3" path="/var/lib/kubelet/pods/45c7d30f-daaa-4834-ac9e-94d51e9aa5c3/volumes"
Sep 15 22:57:05.574133 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:05.574081    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-6e9026"
Sep 15 22:57:05.670542 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:05.670449    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6vsn\" (UniqueName: \"kubernetes.io/projected/2d387e4a-fe84-4a3d-aeee-f8b861944631-kube-api-access-s6vsn\") pod \"2d387e4a-fe84-4a3d-aeee-f8b861944631\" (UID: \"2d387e4a-fe84-4a3d-aeee-f8b861944631\") "
Sep 15 22:57:05.672618 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:05.672584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d387e4a-fe84-4a3d-aeee-f8b861944631-kube-api-access-s6vsn" (OuterVolumeSpecName: "kube-api-access-s6vsn") pod "2d387e4a-fe84-4a3d-aeee-f8b861944631" (UID: "2d387e4a-fe84-4a3d-aeee-f8b861944631"). InnerVolumeSpecName "kube-api-access-s6vsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:57:05.771842 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:05.771808    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6vsn\" (UniqueName: \"kubernetes.io/projected/2d387e4a-fe84-4a3d-aeee-f8b861944631-kube-api-access-s6vsn\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 22:57:06.424790 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:06.424754    2578 generic.go:356] "Generic (PLEG): container finished" podID="2d387e4a-fe84-4a3d-aeee-f8b861944631" containerID="4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a" exitCode=137
Sep 15 22:57:06.424790 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:06.424795    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-6e9026"
Sep 15 22:57:06.425013 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:06.424837    2578 scope.go:117] "RemoveContainer" containerID="4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a"
Sep 15 22:57:06.433677 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:06.433657    2578 scope.go:117] "RemoveContainer" containerID="4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a"
Sep 15 22:57:06.433974 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:57:06.433955    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a\": container with ID starting with 4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a not found: ID does not exist" containerID="4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a"
Sep 15 22:57:06.434018 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:06.433987    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a"} err="failed to get container status \"4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a\": rpc error: code = NotFound desc = could not find container \"4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a\": container with ID starting with 4e348b6ba095bc85e5ef8379bbd14639dc3af1c07d66abd544cf82a7f333e63a not found: ID does not exist"
Sep 15 22:57:06.435782 ip-10-0-128-247 kubenswrapper[2578]: E0915 22:57:06.435758    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7654-6e9026\" is forbidden: User \"system:node:ip-10-0-128-247.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-247.ec2.internal' and this object" podUID="2d387e4a-fe84-4a3d-aeee-f8b861944631" pod="opendatahub/test-curl-7654-6e9026"
Sep 15 22:57:06.763202 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:57:06.763166    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d387e4a-fe84-4a3d-aeee-f8b861944631" path="/var/lib/kubelet/pods/2d387e4a-fe84-4a3d-aeee-f8b861944631/volumes"
Sep 15 22:58:20.750643 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.750601    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"]
Sep 15 22:58:20.751263 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.751155    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d387e4a-fe84-4a3d-aeee-f8b861944631" containerName="test-curl-7654-6e9026"
Sep 15 22:58:20.751263 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.751177    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d387e4a-fe84-4a3d-aeee-f8b861944631" containerName="test-curl-7654-6e9026"
Sep 15 22:58:20.751263 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.751258    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d387e4a-fe84-4a3d-aeee-f8b861944631" containerName="test-curl-7654-6e9026"
Sep 15 22:58:20.754200 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.754132    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.757371 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.757349    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-637a6f-serving-cert\""
Sep 15 22:58:20.763738 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.763710    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"]
Sep 15 22:58:20.859813 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.859776    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcnjs\" (UniqueName: \"kubernetes.io/projected/06bdaa60-a5cc-4292-8be2-3bd923e5d146-kube-api-access-xcnjs\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.859992 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.859827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-metrics-certs\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.859992 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.859946    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06bdaa60-a5cc-4292-8be2-3bd923e5d146-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.859992 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.859983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-maas-api-tls\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.961424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.961310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-maas-api-tls\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.961424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.961363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcnjs\" (UniqueName: \"kubernetes.io/projected/06bdaa60-a5cc-4292-8be2-3bd923e5d146-kube-api-access-xcnjs\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.961424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.961398    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-metrics-certs\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.966644 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.961452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06bdaa60-a5cc-4292-8be2-3bd923e5d146-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.966644 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.962079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06bdaa60-a5cc-4292-8be2-3bd923e5d146-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.966644 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.963833    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-maas-api-tls\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.966644 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.963897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-metrics-certs\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:20.979945 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:20.979921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcnjs\" (UniqueName: \"kubernetes.io/projected/06bdaa60-a5cc-4292-8be2-3bd923e5d146-kube-api-access-xcnjs\") pod \"maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:21.065801 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:21.065696    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:21.339637 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:21.339613    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"]
Sep 15 22:58:21.341583 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:58:21.341549    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06bdaa60_a5cc_4292_8be2_3bd923e5d146.slice/crio-0803cd71a3ca97c80392677abe80b51b96a893b0e1ecd703a96011479ffe6883 WatchSource:0}: Error finding container 0803cd71a3ca97c80392677abe80b51b96a893b0e1ecd703a96011479ffe6883: Status 404 returned error can't find the container with id 0803cd71a3ca97c80392677abe80b51b96a893b0e1ecd703a96011479ffe6883
Sep 15 22:58:21.342931 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:21.342911    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:58:21.781424 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:21.781380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z" event={"ID":"06bdaa60-a5cc-4292-8be2-3bd923e5d146","Type":"ContainerStarted","Data":"0803cd71a3ca97c80392677abe80b51b96a893b0e1ecd703a96011479ffe6883"}
Sep 15 22:58:22.787619 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:22.787583    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z" event={"ID":"06bdaa60-a5cc-4292-8be2-3bd923e5d146","Type":"ContainerStarted","Data":"75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307"}
Sep 15 22:58:22.788006 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:22.787705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:58:22.812732 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:22.812679    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z" podStartSLOduration=2.345465688 podStartE2EDuration="2.812662298s" totalImagesPullingTime="467.19661ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:58:22.811363329 +0000 UTC m=+1842.564987466" watchObservedRunningTime="2026-09-15 22:58:22.812662298 +0000 UTC m=+1842.566286425"
Sep 15 22:58:28.797468 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:58:28.797439    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 22:59:23.757516 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.757433    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"]
Sep 15 22:59:23.760848 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.760824    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.765207 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.765178    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-637a6f\"/\"openshift-service-ca.crt\""
Sep 15 22:59:23.765379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.765244    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-637a6f\"/\"default-dockercfg-4gtwb\""
Sep 15 22:59:23.765379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.765283    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-637a6f\"/\"kube-root-ca.crt\""
Sep 15 22:59:23.765379 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.765245    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-637a6f\"/\"ipp-route-637a6f-kserve-self-signed-certs\""
Sep 15 22:59:23.773313 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.773285    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"]
Sep 15 22:59:23.847021 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.846983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-home\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.847021 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.847033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dadb847-ce85-4126-a074-93af34fbbedd-tls-certs\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.847290 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.847076    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4khbt\" (UniqueName: \"kubernetes.io/projected/1dadb847-ce85-4126-a074-93af34fbbedd-kube-api-access-4khbt\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.847290 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.847145    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-tmp-dir\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.847290 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.847172    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-dshm\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.847290 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.847187    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-model-cache\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.948464 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.948415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-tmp-dir\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.948680 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.948479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-dshm\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.948680 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.948508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-model-cache\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.948680 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.948605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-home\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.948680 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.948670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dadb847-ce85-4126-a074-93af34fbbedd-tls-certs\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.948904 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.948727    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4khbt\" (UniqueName: \"kubernetes.io/projected/1dadb847-ce85-4126-a074-93af34fbbedd-kube-api-access-4khbt\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.948963 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.948944    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-tmp-dir\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.949021 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.948973    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-model-cache\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.949021 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.949007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-home\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.950770 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.950747    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-dshm\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.951035 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.951018    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dadb847-ce85-4126-a074-93af34fbbedd-tls-certs\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:23.957060 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:23.957033    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4khbt\" (UniqueName: \"kubernetes.io/projected/1dadb847-ce85-4126-a074-93af34fbbedd-kube-api-access-4khbt\") pod \"ipp-route-637a6f-kserve-78d8688787-27fn5\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") " pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:24.071877 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:24.071789    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:24.343447 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:24.343364    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"]
Sep 15 22:59:24.346780 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:59:24.346753    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1dadb847_ce85_4126_a074_93af34fbbedd.slice/crio-57d8ffb8bbcd04d0689fa12fed8207c17415831782e9fac8a3ba3a1337434038 WatchSource:0}: Error finding container 57d8ffb8bbcd04d0689fa12fed8207c17415831782e9fac8a3ba3a1337434038: Status 404 returned error can't find the container with id 57d8ffb8bbcd04d0689fa12fed8207c17415831782e9fac8a3ba3a1337434038
Sep 15 22:59:25.069516 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:25.069483    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5" event={"ID":"1dadb847-ce85-4126-a074-93af34fbbedd","Type":"ContainerStarted","Data":"24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6"}
Sep 15 22:59:25.069924 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:25.069526    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5" event={"ID":"1dadb847-ce85-4126-a074-93af34fbbedd","Type":"ContainerStarted","Data":"57d8ffb8bbcd04d0689fa12fed8207c17415831782e9fac8a3ba3a1337434038"}
Sep 15 22:59:25.088043 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:25.087992    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5" podStartSLOduration=2.087975588 podStartE2EDuration="2.087975588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:59:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:59:25.086140665 +0000 UTC m=+1904.839764793" watchObservedRunningTime="2026-09-15 22:59:25.087975588 +0000 UTC m=+1904.841599715"
Sep 15 22:59:34.072537 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:34.072498    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:34.073006 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:34.072735    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:34.085127 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:34.085082    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:34.120722 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:34.120688    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 22:59:49.956972 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:49.956937    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"]
Sep 15 22:59:49.960907 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:49.960882    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:49.963689 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:49.963661    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-10ec22-openshift-default-dockercfg-l798h\""
Sep 15 22:59:49.973456 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:49.973428    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"]
Sep 15 22:59:50.081179 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.081140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-certs\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.081355 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.081213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9p9p\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-kube-api-access-d9p9p\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.081355 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.081240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-socket\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.081355 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.081267    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-credential-socket\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.081463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.081357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-token\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.081463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.081398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-data\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.081463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.081432    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.081463 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.081461    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-envoy\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.081590 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.081483    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-podinfo\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182262 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9p9p\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-kube-api-access-d9p9p\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182451 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-socket\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182451 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-credential-socket\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182451 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-token\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182451 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182381    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-data\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182451 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182451 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-envoy\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182780 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182470    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-podinfo\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182780 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-certs\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182780 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-socket\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182780 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182723    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-credential-socket\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182995 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182774    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-data\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.182995 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.182896    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-certs\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.183382 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.183351    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.184837 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.184812    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-envoy\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.185250 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.185229    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-podinfo\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.192393 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.192368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-token\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.192509 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.192455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9p9p\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-kube-api-access-d9p9p\") pod \"e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") " pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.272634 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.272535    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:50.550350 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.550281    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"]
Sep 15 22:59:50.552694 ip-10-0-128-247 kubenswrapper[2578]: W0915 22:59:50.552668    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb838f018_d61f_49e5_9fdb_991bcb74bbe7.slice/crio-26adae26e0c6eaf8de48d54a9eab711465b4fb1354d0ae14924c2c894dd14dde WatchSource:0}: Error finding container 26adae26e0c6eaf8de48d54a9eab711465b4fb1354d0ae14924c2c894dd14dde: Status 404 returned error can't find the container with id 26adae26e0c6eaf8de48d54a9eab711465b4fb1354d0ae14924c2c894dd14dde
Sep 15 22:59:50.555085 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.555052    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:59:50.555202 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.555133    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:59:50.555202 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:50.555164    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 22:59:51.190945 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:51.190907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" event={"ID":"b838f018-d61f-49e5-9fdb-991bcb74bbe7","Type":"ContainerStarted","Data":"d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495"}
Sep 15 22:59:51.191345 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:51.190989    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" event={"ID":"b838f018-d61f-49e5-9fdb-991bcb74bbe7","Type":"ContainerStarted","Data":"26adae26e0c6eaf8de48d54a9eab711465b4fb1354d0ae14924c2c894dd14dde"}
Sep 15 22:59:51.210811 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:51.210750    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" podStartSLOduration=2.210729303 podStartE2EDuration="2.210729303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:59:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:59:51.208775917 +0000 UTC m=+1930.962400045" watchObservedRunningTime="2026-09-15 22:59:51.210729303 +0000 UTC m=+1930.964353431"
Sep 15 22:59:51.273076 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:51.273036    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:51.278428 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:51.278401    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:52.195975 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:52.195945    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 22:59:52.197142 ip-10-0-128-247 kubenswrapper[2578]: I0915 22:59:52.197122    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 23:00:00.135507 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.135469    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x"]
Sep 15 23:00:00.139516 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.139489    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph"]
Sep 15 23:00:00.139732 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.139709    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x"
Sep 15 23:00:00.143055 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.143028    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hnj7x\""
Sep 15 23:00:00.148807 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.148776    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph"
Sep 15 23:00:00.154119 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.152835    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x"]
Sep 15 23:00:00.155772 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.155738    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph"]
Sep 15 23:00:00.167412 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.167387    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x"
Sep 15 23:00:00.177540 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.177513    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph"
Sep 15 23:00:00.447314 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.447279    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x"]
Sep 15 23:00:00.451124 ip-10-0-128-247 kubenswrapper[2578]: W0915 23:00:00.451072    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0991a553_ad38_4115_bb53_8ebf7efac73e.slice/crio-93d7a6cae4aa42fe4df36bbbdbea654c1a3f1c1cf71f265c04f739eee26e1371 WatchSource:0}: Error finding container 93d7a6cae4aa42fe4df36bbbdbea654c1a3f1c1cf71f265c04f739eee26e1371: Status 404 returned error can't find the container with id 93d7a6cae4aa42fe4df36bbbdbea654c1a3f1c1cf71f265c04f739eee26e1371
Sep 15 23:00:00.458384 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:00.458345    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph"]
Sep 15 23:00:00.459365 ip-10-0-128-247 kubenswrapper[2578]: W0915 23:00:00.459328    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1c6aa9_8f31_4758_bc34_c50c217a13f4.slice/crio-101167b059ba53f1121742eda8621de18e907d51bf561b2e6c2a73e6576add0c WatchSource:0}: Error finding container 101167b059ba53f1121742eda8621de18e907d51bf561b2e6c2a73e6576add0c: Status 404 returned error can't find the container with id 101167b059ba53f1121742eda8621de18e907d51bf561b2e6c2a73e6576add0c
Sep 15 23:00:01.242696 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:01.242637    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph" event={"ID":"0e1c6aa9-8f31-4758-bc34-c50c217a13f4","Type":"ContainerStarted","Data":"ccfcf13f431dd29e944fba3e5fa811acfafcd94724f4759891e5a1dd36e62cae"}
Sep 15 23:00:01.243439 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:01.242715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph" event={"ID":"0e1c6aa9-8f31-4758-bc34-c50c217a13f4","Type":"ContainerStarted","Data":"101167b059ba53f1121742eda8621de18e907d51bf561b2e6c2a73e6576add0c"}
Sep 15 23:00:01.244217 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:01.244187    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x" event={"ID":"0991a553-ad38-4115-bb53-8ebf7efac73e","Type":"ContainerStarted","Data":"8f16ae0142b6c260ea1fa1eef3a338af7d04418053ef65ffcb25309ab07854fd"}
Sep 15 23:00:01.244326 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:01.244238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x" event={"ID":"0991a553-ad38-4115-bb53-8ebf7efac73e","Type":"ContainerStarted","Data":"93d7a6cae4aa42fe4df36bbbdbea654c1a3f1c1cf71f265c04f739eee26e1371"}
Sep 15 23:00:01.258949 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:01.258896    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph" podStartSLOduration=1.2588781519999999 podStartE2EDuration="1.258878152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 23:00:01.257214191 +0000 UTC m=+1941.010838320" watchObservedRunningTime="2026-09-15 23:00:01.258878152 +0000 UTC m=+1941.012502280"
Sep 15 23:00:01.272492 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:01.272433    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x" podStartSLOduration=1.272412257 podStartE2EDuration="1.272412257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 23:00:01.272212312 +0000 UTC m=+1941.025836452" watchObservedRunningTime="2026-09-15 23:00:01.272412257 +0000 UTC m=+1941.026036385"
Sep 15 23:00:02.253530 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:02.253492    2578 generic.go:356] "Generic (PLEG): container finished" podID="0991a553-ad38-4115-bb53-8ebf7efac73e" containerID="8f16ae0142b6c260ea1fa1eef3a338af7d04418053ef65ffcb25309ab07854fd" exitCode=0
Sep 15 23:00:02.254196 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:02.253814    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x" event={"ID":"0991a553-ad38-4115-bb53-8ebf7efac73e","Type":"ContainerDied","Data":"8f16ae0142b6c260ea1fa1eef3a338af7d04418053ef65ffcb25309ab07854fd"}
Sep 15 23:00:02.256460 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:02.256427    2578 generic.go:356] "Generic (PLEG): container finished" podID="0e1c6aa9-8f31-4758-bc34-c50c217a13f4" containerID="ccfcf13f431dd29e944fba3e5fa811acfafcd94724f4759891e5a1dd36e62cae" exitCode=0
Sep 15 23:00:02.256582 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:02.256493    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph" event={"ID":"0e1c6aa9-8f31-4758-bc34-c50c217a13f4","Type":"ContainerDied","Data":"ccfcf13f431dd29e944fba3e5fa811acfafcd94724f4759891e5a1dd36e62cae"}
Sep 15 23:00:03.426055 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:03.426011    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph"
Sep 15 23:00:03.451143 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:03.451114    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x"
Sep 15 23:00:04.267939 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:04.267907    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x"
Sep 15 23:00:04.267939 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:04.267931    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x" event={"ID":"0991a553-ad38-4115-bb53-8ebf7efac73e","Type":"ContainerDied","Data":"93d7a6cae4aa42fe4df36bbbdbea654c1a3f1c1cf71f265c04f739eee26e1371"}
Sep 15 23:00:04.268221 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:04.267987    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93d7a6cae4aa42fe4df36bbbdbea654c1a3f1c1cf71f265c04f739eee26e1371"
Sep 15 23:00:04.269373 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:04.269350    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph"
Sep 15 23:00:04.269506 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:04.269373    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph" event={"ID":"0e1c6aa9-8f31-4758-bc34-c50c217a13f4","Type":"ContainerDied","Data":"101167b059ba53f1121742eda8621de18e907d51bf561b2e6c2a73e6576add0c"}
Sep 15 23:00:04.269506 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:04.269436    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="101167b059ba53f1121742eda8621de18e907d51bf561b2e6c2a73e6576add0c"
Sep 15 23:00:12.310158 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:12.310117    2578 generic.go:356] "Generic (PLEG): container finished" podID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerID="d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495" exitCode=0
Sep 15 23:00:12.310611 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:12.310182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" event={"ID":"b838f018-d61f-49e5-9fdb-991bcb74bbe7","Type":"ContainerDied","Data":"d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495"}
Sep 15 23:00:12.310656 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:12.310612    2578 scope.go:117] "RemoveContainer" containerID="d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495"
Sep 15 23:00:12.311783 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:12.311754    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 23:00:12.311851 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:12.311822    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 23:00:12.311887 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:12.311856    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 23:00:13.321212 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:13.321180    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" event={"ID":"b838f018-d61f-49e5-9fdb-991bcb74bbe7","Type":"ContainerStarted","Data":"95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611"}
Sep 15 23:00:14.273324 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:14.273288    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 23:00:14.273631 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:14.273606    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 23:00:14.273710 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:14.273670    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 23:00:15.273965 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:15.273928    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 23:00:15.274371 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:15.273983    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 23:00:16.273812 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:16.273768    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 23:00:16.273997 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:16.273836    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 23:00:17.273248 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:17.273204    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 23:00:17.273445 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:17.273282    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 23:00:18.273085 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:18.273048    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 23:00:18.273512 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:18.273121    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 23:00:19.278066 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:19.278033    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 23:00:19.278495 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:19.278336    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 23:00:19.279198 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:19.279177    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 23:00:31.858257 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:31.858202    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"]
Sep 15 23:00:31.858894 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:31.858456    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy" containerID="cri-o://95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611" gracePeriod=30
Sep 15 23:00:34.281186 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:34.281149    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 23:00:34.281586 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:34.281226    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 23:00:37.104270 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.104245    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 23:00:37.240192 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240154    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-credential-socket\") pod \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") "
Sep 15 23:00:37.240192 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240191    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-envoy\") pod \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") "
Sep 15 23:00:37.240465 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240214    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istiod-ca-cert\") pod \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") "
Sep 15 23:00:37.240465 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240238    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-token\") pod \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") "
Sep 15 23:00:37.240465 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240332    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-certs\") pod \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") "
Sep 15 23:00:37.240465 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240379    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-podinfo\") pod \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") "
Sep 15 23:00:37.240465 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240408    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9p9p\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-kube-api-access-d9p9p\") pod \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") "
Sep 15 23:00:37.240465 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240431    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-socket\") pod \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") "
Sep 15 23:00:37.240465 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240459    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-data\") pod \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\" (UID: \"b838f018-d61f-49e5-9fdb-991bcb74bbe7\") "
Sep 15 23:00:37.240844 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240519    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b838f018-d61f-49e5-9fdb-991bcb74bbe7" (UID: "b838f018-d61f-49e5-9fdb-991bcb74bbe7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:00:37.240844 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240688    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b838f018-d61f-49e5-9fdb-991bcb74bbe7" (UID: "b838f018-d61f-49e5-9fdb-991bcb74bbe7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:00:37.240844 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240798    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b838f018-d61f-49e5-9fdb-991bcb74bbe7" (UID: "b838f018-d61f-49e5-9fdb-991bcb74bbe7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:00:37.240979 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240864    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-data" (OuterVolumeSpecName: "istio-data") pod "b838f018-d61f-49e5-9fdb-991bcb74bbe7" (UID: "b838f018-d61f-49e5-9fdb-991bcb74bbe7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:00:37.240979 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.240890    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b838f018-d61f-49e5-9fdb-991bcb74bbe7" (UID: "b838f018-d61f-49e5-9fdb-991bcb74bbe7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:00:37.241128 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.241112    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:37.241201 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.241136    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:37.241201 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.241151    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:37.241201 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.241165    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:37.241201 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.241178    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:37.242493 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.242471    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b838f018-d61f-49e5-9fdb-991bcb74bbe7" (UID: "b838f018-d61f-49e5-9fdb-991bcb74bbe7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:00:37.242705 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.242681    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-token" (OuterVolumeSpecName: "istio-token") pod "b838f018-d61f-49e5-9fdb-991bcb74bbe7" (UID: "b838f018-d61f-49e5-9fdb-991bcb74bbe7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:00:37.242901 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.242882    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b838f018-d61f-49e5-9fdb-991bcb74bbe7" (UID: "b838f018-d61f-49e5-9fdb-991bcb74bbe7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 23:00:37.242961 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.242911    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-kube-api-access-d9p9p" (OuterVolumeSpecName: "kube-api-access-d9p9p") pod "b838f018-d61f-49e5-9fdb-991bcb74bbe7" (UID: "b838f018-d61f-49e5-9fdb-991bcb74bbe7"). InnerVolumeSpecName "kube-api-access-d9p9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:00:37.341686 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.341648    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:37.341686 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.341684    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:37.341686 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.341693    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b838f018-d61f-49e5-9fdb-991bcb74bbe7-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:37.341933 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.341703    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9p9p\" (UniqueName: \"kubernetes.io/projected/b838f018-d61f-49e5-9fdb-991bcb74bbe7-kube-api-access-d9p9p\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:37.436674 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.436629    2578 generic.go:356] "Generic (PLEG): container finished" podID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerID="95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611" exitCode=0
Sep 15 23:00:37.436852 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.436690    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"
Sep 15 23:00:37.436852 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.436711    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" event={"ID":"b838f018-d61f-49e5-9fdb-991bcb74bbe7","Type":"ContainerDied","Data":"95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611"}
Sep 15 23:00:37.436852 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.436768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc" event={"ID":"b838f018-d61f-49e5-9fdb-991bcb74bbe7","Type":"ContainerDied","Data":"26adae26e0c6eaf8de48d54a9eab711465b4fb1354d0ae14924c2c894dd14dde"}
Sep 15 23:00:37.436852 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.436791    2578 scope.go:117] "RemoveContainer" containerID="95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611"
Sep 15 23:00:37.446272 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.446255    2578 scope.go:117] "RemoveContainer" containerID="d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495"
Sep 15 23:00:37.454607 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.454584    2578 scope.go:117] "RemoveContainer" containerID="95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611"
Sep 15 23:00:37.454904 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:00:37.454883    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611\": container with ID starting with 95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611 not found: ID does not exist" containerID="95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611"
Sep 15 23:00:37.454948 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.454912    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611"} err="failed to get container status \"95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611\": rpc error: code = NotFound desc = could not find container \"95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611\": container with ID starting with 95c26a579a36bce4d12a37b09da7cd515b8420c48933382d324f37af8bdc3611 not found: ID does not exist"
Sep 15 23:00:37.454948 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.454931    2578 scope.go:117] "RemoveContainer" containerID="d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495"
Sep 15 23:00:37.455212 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:00:37.455194    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495\": container with ID starting with d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495 not found: ID does not exist" containerID="d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495"
Sep 15 23:00:37.455279 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.455222    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495"} err="failed to get container status \"d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495\": rpc error: code = NotFound desc = could not find container \"d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495\": container with ID starting with d0393910e1ee0e9cd40a25e0f74409e3f59e43c3a0936e985692c1ae96c02495 not found: ID does not exist"
Sep 15 23:00:37.459032 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.459009    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"]
Sep 15 23:00:37.464647 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:37.464619    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-10ec22-openshift-default-6f946cb55d-m5pjc"]
Sep 15 23:00:38.763351 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:38.763321    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" path="/var/lib/kubelet/pods/b838f018-d61f-49e5-9fdb-991bcb74bbe7/volumes"
Sep 15 23:00:51.031408 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.031359    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"]
Sep 15 23:00:51.031860 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.031601    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z" podUID="06bdaa60-a5cc-4292-8be2-3bd923e5d146" containerName="maas-api" containerID="cri-o://75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307" gracePeriod=30
Sep 15 23:00:51.271274 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.271249    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 23:00:51.372443 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.372350    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcnjs\" (UniqueName: \"kubernetes.io/projected/06bdaa60-a5cc-4292-8be2-3bd923e5d146-kube-api-access-xcnjs\") pod \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") "
Sep 15 23:00:51.372443 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.372389    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06bdaa60-a5cc-4292-8be2-3bd923e5d146-openshift-service-ca\") pod \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") "
Sep 15 23:00:51.372443 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.372407    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-maas-api-tls\") pod \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") "
Sep 15 23:00:51.372745 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.372452    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-metrics-certs\") pod \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\" (UID: \"06bdaa60-a5cc-4292-8be2-3bd923e5d146\") "
Sep 15 23:00:51.372878 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.372851    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06bdaa60-a5cc-4292-8be2-3bd923e5d146-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06bdaa60-a5cc-4292-8be2-3bd923e5d146" (UID: "06bdaa60-a5cc-4292-8be2-3bd923e5d146"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:00:51.374720 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.374684    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "06bdaa60-a5cc-4292-8be2-3bd923e5d146" (UID: "06bdaa60-a5cc-4292-8be2-3bd923e5d146"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:00:51.374841 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.374698    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06bdaa60-a5cc-4292-8be2-3bd923e5d146" (UID: "06bdaa60-a5cc-4292-8be2-3bd923e5d146"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:00:51.374841 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.374727    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06bdaa60-a5cc-4292-8be2-3bd923e5d146-kube-api-access-xcnjs" (OuterVolumeSpecName: "kube-api-access-xcnjs") pod "06bdaa60-a5cc-4292-8be2-3bd923e5d146" (UID: "06bdaa60-a5cc-4292-8be2-3bd923e5d146"). InnerVolumeSpecName "kube-api-access-xcnjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:00:51.473617 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.473583    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcnjs\" (UniqueName: \"kubernetes.io/projected/06bdaa60-a5cc-4292-8be2-3bd923e5d146-kube-api-access-xcnjs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:51.473617 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.473613    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06bdaa60-a5cc-4292-8be2-3bd923e5d146-openshift-service-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:51.473617 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.473624    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-maas-api-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:51.473865 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.473633    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06bdaa60-a5cc-4292-8be2-3bd923e5d146-metrics-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:00:51.509021 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.508989    2578 generic.go:356] "Generic (PLEG): container finished" podID="06bdaa60-a5cc-4292-8be2-3bd923e5d146" containerID="75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307" exitCode=0
Sep 15 23:00:51.509208 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.509042    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"
Sep 15 23:00:51.509208 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.509078    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z" event={"ID":"06bdaa60-a5cc-4292-8be2-3bd923e5d146","Type":"ContainerDied","Data":"75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307"}
Sep 15 23:00:51.509208 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.509167    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z" event={"ID":"06bdaa60-a5cc-4292-8be2-3bd923e5d146","Type":"ContainerDied","Data":"0803cd71a3ca97c80392677abe80b51b96a893b0e1ecd703a96011479ffe6883"}
Sep 15 23:00:51.509208 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.509198    2578 scope.go:117] "RemoveContainer" containerID="75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307"
Sep 15 23:00:51.518678 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.518660    2578 scope.go:117] "RemoveContainer" containerID="75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307"
Sep 15 23:00:51.518972 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:00:51.518954    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307\": container with ID starting with 75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307 not found: ID does not exist" containerID="75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307"
Sep 15 23:00:51.519018 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.518981    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307"} err="failed to get container status \"75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307\": rpc error: code = NotFound desc = could not find container \"75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307\": container with ID starting with 75bbc8dbb21955b8c0b59622c6074103060a5deb8ae2ffae2f1969d93e83f307 not found: ID does not exist"
Sep 15 23:00:51.531356 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.531332    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"]
Sep 15 23:00:51.534972 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:51.534951    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-637a6f-69b94c59bf-7rs9z"]
Sep 15 23:00:52.763050 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:52.763013    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06bdaa60-a5cc-4292-8be2-3bd923e5d146" path="/var/lib/kubelet/pods/06bdaa60-a5cc-4292-8be2-3bd923e5d146/volumes"
Sep 15 23:00:56.600536 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:00:56.600502    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"]
Sep 15 23:01:11.620380 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.620334    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5" podUID="1dadb847-ce85-4126-a074-93af34fbbedd" containerName="main" containerID="cri-o://24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6" gracePeriod=45
Sep 15 23:01:11.864917 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.864891    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 23:01:11.956733 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.956700    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dadb847-ce85-4126-a074-93af34fbbedd-tls-certs\") pod \"1dadb847-ce85-4126-a074-93af34fbbedd\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") "
Sep 15 23:01:11.956928 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.956761    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-home\") pod \"1dadb847-ce85-4126-a074-93af34fbbedd\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") "
Sep 15 23:01:11.956928 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.956789    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4khbt\" (UniqueName: \"kubernetes.io/projected/1dadb847-ce85-4126-a074-93af34fbbedd-kube-api-access-4khbt\") pod \"1dadb847-ce85-4126-a074-93af34fbbedd\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") "
Sep 15 23:01:11.956928 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.956859    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-tmp-dir\") pod \"1dadb847-ce85-4126-a074-93af34fbbedd\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") "
Sep 15 23:01:11.956928 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.956910    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-model-cache\") pod \"1dadb847-ce85-4126-a074-93af34fbbedd\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") "
Sep 15 23:01:11.957200 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.956937    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-dshm\") pod \"1dadb847-ce85-4126-a074-93af34fbbedd\" (UID: \"1dadb847-ce85-4126-a074-93af34fbbedd\") "
Sep 15 23:01:11.957200 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.957123    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-home" (OuterVolumeSpecName: "home") pod "1dadb847-ce85-4126-a074-93af34fbbedd" (UID: "1dadb847-ce85-4126-a074-93af34fbbedd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:01:11.957316 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.957206    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1dadb847-ce85-4126-a074-93af34fbbedd" (UID: "1dadb847-ce85-4126-a074-93af34fbbedd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:01:11.957316 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.957300    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:01:11.957406 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.957319    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:01:11.957449 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.957400    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-model-cache" (OuterVolumeSpecName: "model-cache") pod "1dadb847-ce85-4126-a074-93af34fbbedd" (UID: "1dadb847-ce85-4126-a074-93af34fbbedd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:01:11.959138 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.959086    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-dshm" (OuterVolumeSpecName: "dshm") pod "1dadb847-ce85-4126-a074-93af34fbbedd" (UID: "1dadb847-ce85-4126-a074-93af34fbbedd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:01:11.959266 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.959157    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dadb847-ce85-4126-a074-93af34fbbedd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1dadb847-ce85-4126-a074-93af34fbbedd" (UID: "1dadb847-ce85-4126-a074-93af34fbbedd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:01:11.959266 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:11.959237    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dadb847-ce85-4126-a074-93af34fbbedd-kube-api-access-4khbt" (OuterVolumeSpecName: "kube-api-access-4khbt") pod "1dadb847-ce85-4126-a074-93af34fbbedd" (UID: "1dadb847-ce85-4126-a074-93af34fbbedd"). InnerVolumeSpecName "kube-api-access-4khbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:01:12.058781 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.058740    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:01:12.058781 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.058772    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dadb847-ce85-4126-a074-93af34fbbedd-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:01:12.058781 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.058781    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dadb847-ce85-4126-a074-93af34fbbedd-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:01:12.058781 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.058791    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4khbt\" (UniqueName: \"kubernetes.io/projected/1dadb847-ce85-4126-a074-93af34fbbedd-kube-api-access-4khbt\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:01:12.609012 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.608972    2578 generic.go:356] "Generic (PLEG): container finished" podID="1dadb847-ce85-4126-a074-93af34fbbedd" containerID="24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6" exitCode=0
Sep 15 23:01:12.609245 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.609022    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"
Sep 15 23:01:12.609245 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.609065    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5" event={"ID":"1dadb847-ce85-4126-a074-93af34fbbedd","Type":"ContainerDied","Data":"24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6"}
Sep 15 23:01:12.609245 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.609133    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5" event={"ID":"1dadb847-ce85-4126-a074-93af34fbbedd","Type":"ContainerDied","Data":"57d8ffb8bbcd04d0689fa12fed8207c17415831782e9fac8a3ba3a1337434038"}
Sep 15 23:01:12.609245 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.609156    2578 scope.go:117] "RemoveContainer" containerID="24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6"
Sep 15 23:01:12.617592 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.617575    2578 scope.go:117] "RemoveContainer" containerID="24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6"
Sep 15 23:01:12.617886 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:01:12.617866    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6\": container with ID starting with 24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6 not found: ID does not exist" containerID="24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6"
Sep 15 23:01:12.617984 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.617891    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6"} err="failed to get container status \"24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6\": rpc error: code = NotFound desc = could not find container \"24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6\": container with ID starting with 24c30c3915a8f1ddfedbc965dae8fb495fdc8078f2b0bd1cef9d0999af267cc6 not found: ID does not exist"
Sep 15 23:01:12.631619 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.631595    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"]
Sep 15 23:01:12.635657 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.635635    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-637a6f/ipp-route-637a6f-kserve-78d8688787-27fn5"]
Sep 15 23:01:12.762072 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:01:12.762035    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dadb847-ce85-4126-a074-93af34fbbedd" path="/var/lib/kubelet/pods/1dadb847-ce85-4126-a074-93af34fbbedd/volumes"
Sep 15 23:02:13.513169 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:13.513069    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph"]
Sep 15 23:02:13.516428 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:13.516403    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0016d4-29825220-zk5ph"]
Sep 15 23:02:14.763015 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:14.762982    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e1c6aa9-8f31-4758-bc34-c50c217a13f4" path="/var/lib/kubelet/pods/0e1c6aa9-8f31-4758-bc34-c50c217a13f4/volumes"
Sep 15 23:02:20.943574 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:20.943538    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"]
Sep 15 23:02:20.943983 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:20.943741    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy" containerID="cri-o://26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b" gracePeriod=30
Sep 15 23:02:25.149015 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:25.148976    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 23:02:25.149423 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:25.149048    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 23:02:26.192864 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.192837    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 23:02:26.245163 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245061    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67pt5\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-kube-api-access-67pt5\") pod \"09e12005-9215-41e3-8808-c54d4bfca288\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") "
Sep 15 23:02:26.245163 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245133    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e12005-9215-41e3-8808-c54d4bfca288-istiod-ca-cert\") pod \"09e12005-9215-41e3-8808-c54d4bfca288\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") "
Sep 15 23:02:26.245163 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245160    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-certs\") pod \"09e12005-9215-41e3-8808-c54d4bfca288\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") "
Sep 15 23:02:26.245451 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245206    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-socket\") pod \"09e12005-9215-41e3-8808-c54d4bfca288\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") "
Sep 15 23:02:26.245451 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245239    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-credential-socket\") pod \"09e12005-9215-41e3-8808-c54d4bfca288\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") "
Sep 15 23:02:26.245451 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245257    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-istio-token\") pod \"09e12005-9215-41e3-8808-c54d4bfca288\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") "
Sep 15 23:02:26.245451 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245284    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-data\") pod \"09e12005-9215-41e3-8808-c54d4bfca288\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") "
Sep 15 23:02:26.245451 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245298    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-envoy\") pod \"09e12005-9215-41e3-8808-c54d4bfca288\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") "
Sep 15 23:02:26.245451 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245343    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e12005-9215-41e3-8808-c54d4bfca288-istio-podinfo\") pod \"09e12005-9215-41e3-8808-c54d4bfca288\" (UID: \"09e12005-9215-41e3-8808-c54d4bfca288\") "
Sep 15 23:02:26.245774 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245448    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09e12005-9215-41e3-8808-c54d4bfca288" (UID: "09e12005-9215-41e3-8808-c54d4bfca288"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:26.245774 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245586    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09e12005-9215-41e3-8808-c54d4bfca288" (UID: "09e12005-9215-41e3-8808-c54d4bfca288"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:26.245774 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245635    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09e12005-9215-41e3-8808-c54d4bfca288-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09e12005-9215-41e3-8808-c54d4bfca288" (UID: "09e12005-9215-41e3-8808-c54d4bfca288"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:02:26.245774 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245740    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:26.245774 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245759    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:26.245774 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245752    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-data" (OuterVolumeSpecName: "istio-data") pod "09e12005-9215-41e3-8808-c54d4bfca288" (UID: "09e12005-9215-41e3-8808-c54d4bfca288"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:26.246084 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.245810    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09e12005-9215-41e3-8808-c54d4bfca288" (UID: "09e12005-9215-41e3-8808-c54d4bfca288"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:26.247443 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.247418    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-kube-api-access-67pt5" (OuterVolumeSpecName: "kube-api-access-67pt5") pod "09e12005-9215-41e3-8808-c54d4bfca288" (UID: "09e12005-9215-41e3-8808-c54d4bfca288"). InnerVolumeSpecName "kube-api-access-67pt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:02:26.247565 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.247462    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09e12005-9215-41e3-8808-c54d4bfca288-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09e12005-9215-41e3-8808-c54d4bfca288" (UID: "09e12005-9215-41e3-8808-c54d4bfca288"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 23:02:26.247565 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.247494    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-istio-token" (OuterVolumeSpecName: "istio-token") pod "09e12005-9215-41e3-8808-c54d4bfca288" (UID: "09e12005-9215-41e3-8808-c54d4bfca288"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:02:26.247644 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.247572    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09e12005-9215-41e3-8808-c54d4bfca288" (UID: "09e12005-9215-41e3-8808-c54d4bfca288"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:26.346411 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.346372    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:26.346411 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.346403    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:26.346411 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.346412    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e12005-9215-41e3-8808-c54d4bfca288-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:26.346411 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.346421    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67pt5\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-kube-api-access-67pt5\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:26.346688 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.346432    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e12005-9215-41e3-8808-c54d4bfca288-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:26.346688 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.346441    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e12005-9215-41e3-8808-c54d4bfca288-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:26.346688 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.346449    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09e12005-9215-41e3-8808-c54d4bfca288-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:26.951040 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.951007    2578 generic.go:356] "Generic (PLEG): container finished" podID="09e12005-9215-41e3-8808-c54d4bfca288" containerID="26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b" exitCode=0
Sep 15 23:02:26.951201 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.951058    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"
Sep 15 23:02:26.951201 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.951059    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" event={"ID":"09e12005-9215-41e3-8808-c54d4bfca288","Type":"ContainerDied","Data":"26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b"}
Sep 15 23:02:26.951291 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.951193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz" event={"ID":"09e12005-9215-41e3-8808-c54d4bfca288","Type":"ContainerDied","Data":"3bef1139efc533b717aa9b4c5db30d240dca75e8af776a9b269609d9617f1422"}
Sep 15 23:02:26.951291 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.951228    2578 scope.go:117] "RemoveContainer" containerID="26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b"
Sep 15 23:02:26.959692 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.959672    2578 scope.go:117] "RemoveContainer" containerID="32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe"
Sep 15 23:02:26.967756 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.967736    2578 scope.go:117] "RemoveContainer" containerID="26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b"
Sep 15 23:02:26.968284 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:02:26.968071    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b\": container with ID starting with 26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b not found: ID does not exist" containerID="26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b"
Sep 15 23:02:26.968400 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.968298    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b"} err="failed to get container status \"26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b\": rpc error: code = NotFound desc = could not find container \"26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b\": container with ID starting with 26832dba4baf3048b70fdc0c6666f8b85346c0849e57e039b5eb7491cef66b8b not found: ID does not exist"
Sep 15 23:02:26.968400 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.968327    2578 scope.go:117] "RemoveContainer" containerID="32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe"
Sep 15 23:02:26.968689 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:02:26.968658    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe\": container with ID starting with 32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe not found: ID does not exist" containerID="32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe"
Sep 15 23:02:26.968754 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.968700    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe"} err="failed to get container status \"32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe\": rpc error: code = NotFound desc = could not find container \"32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe\": container with ID starting with 32e8996923bb0ba882bb7a8a6f0646ed796bde1448d8b641775564053f34fbfe not found: ID does not exist"
Sep 15 23:02:26.971135 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.971114    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"]
Sep 15 23:02:26.974062 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:26.974040    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0016d4-openshift-default-65b5b9c9c4-pd5qz"]
Sep 15 23:02:28.763471 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:28.763441    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09e12005-9215-41e3-8808-c54d4bfca288" path="/var/lib/kubelet/pods/09e12005-9215-41e3-8808-c54d4bfca288/volumes"
Sep 15 23:02:31.937328 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:31.937289    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"]
Sep 15 23:02:31.937757 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:31.937504    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht" podUID="a635b117-00e1-4357-a8ef-e2a0caf593d8" containerName="maas-api" containerID="cri-o://cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84" gracePeriod=30
Sep 15 23:02:31.961472 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:31.961414    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x"]
Sep 15 23:02:31.964748 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:31.964722    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-92482b-29825220-9nr4x"]
Sep 15 23:02:32.175287 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.175260    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 23:02:32.199514 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.199417    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-maas-api-tls\") pod \"a635b117-00e1-4357-a8ef-e2a0caf593d8\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") "
Sep 15 23:02:32.199514 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.199477    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k797n\" (UniqueName: \"kubernetes.io/projected/a635b117-00e1-4357-a8ef-e2a0caf593d8-kube-api-access-k797n\") pod \"a635b117-00e1-4357-a8ef-e2a0caf593d8\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") "
Sep 15 23:02:32.199514 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.199511    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a635b117-00e1-4357-a8ef-e2a0caf593d8-openshift-service-ca\") pod \"a635b117-00e1-4357-a8ef-e2a0caf593d8\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") "
Sep 15 23:02:32.199771 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.199669    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-metrics-certs\") pod \"a635b117-00e1-4357-a8ef-e2a0caf593d8\" (UID: \"a635b117-00e1-4357-a8ef-e2a0caf593d8\") "
Sep 15 23:02:32.199914 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.199883    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a635b117-00e1-4357-a8ef-e2a0caf593d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a635b117-00e1-4357-a8ef-e2a0caf593d8" (UID: "a635b117-00e1-4357-a8ef-e2a0caf593d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:02:32.201925 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.201892    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a635b117-00e1-4357-a8ef-e2a0caf593d8" (UID: "a635b117-00e1-4357-a8ef-e2a0caf593d8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:02:32.202054 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.201944    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a635b117-00e1-4357-a8ef-e2a0caf593d8-kube-api-access-k797n" (OuterVolumeSpecName: "kube-api-access-k797n") pod "a635b117-00e1-4357-a8ef-e2a0caf593d8" (UID: "a635b117-00e1-4357-a8ef-e2a0caf593d8"). InnerVolumeSpecName "kube-api-access-k797n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:02:32.202054 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.201991    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a635b117-00e1-4357-a8ef-e2a0caf593d8" (UID: "a635b117-00e1-4357-a8ef-e2a0caf593d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:02:32.301191 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.301148    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-maas-api-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:32.301191 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.301188    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k797n\" (UniqueName: \"kubernetes.io/projected/a635b117-00e1-4357-a8ef-e2a0caf593d8-kube-api-access-k797n\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:32.301385 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.301202    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a635b117-00e1-4357-a8ef-e2a0caf593d8-openshift-service-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:32.301385 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.301214    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a635b117-00e1-4357-a8ef-e2a0caf593d8-metrics-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:32.763188 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.763152    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0991a553-ad38-4115-bb53-8ebf7efac73e" path="/var/lib/kubelet/pods/0991a553-ad38-4115-bb53-8ebf7efac73e/volumes"
Sep 15 23:02:32.982030 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.981995    2578 generic.go:356] "Generic (PLEG): container finished" podID="a635b117-00e1-4357-a8ef-e2a0caf593d8" containerID="cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84" exitCode=0
Sep 15 23:02:32.982030 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.982033    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"
Sep 15 23:02:32.982537 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.982078    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht" event={"ID":"a635b117-00e1-4357-a8ef-e2a0caf593d8","Type":"ContainerDied","Data":"cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84"}
Sep 15 23:02:32.982537 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.982150    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht" event={"ID":"a635b117-00e1-4357-a8ef-e2a0caf593d8","Type":"ContainerDied","Data":"876b2d1c3daa9d336c051f620afa14dc4227ff7e8b917f1c38c57547fedd3ddd"}
Sep 15 23:02:32.982537 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.982176    2578 scope.go:117] "RemoveContainer" containerID="cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84"
Sep 15 23:02:32.990686 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.990666    2578 scope.go:117] "RemoveContainer" containerID="cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84"
Sep 15 23:02:32.990968 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:02:32.990948    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84\": container with ID starting with cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84 not found: ID does not exist" containerID="cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84"
Sep 15 23:02:32.991051 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:32.990976    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84"} err="failed to get container status \"cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84\": rpc error: code = NotFound desc = could not find container \"cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84\": container with ID starting with cfea8a78fb71f4e0fb7fecd24e840670b87b805fc62a4e0b841e862c32bdec84 not found: ID does not exist"
Sep 15 23:02:33.018320 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:33.018242    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"]
Sep 15 23:02:33.034111 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:33.032288    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht"]
Sep 15 23:02:34.763027 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:34.762990    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a635b117-00e1-4357-a8ef-e2a0caf593d8" path="/var/lib/kubelet/pods/a635b117-00e1-4357-a8ef-e2a0caf593d8/volumes"
Sep 15 23:02:35.154439 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:35.154328    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-92482b-5db6cc795f-s49ht" podUID="a635b117-00e1-4357-a8ef-e2a0caf593d8" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.50:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 15 23:02:38.663390 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:38.663353    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"]
Sep 15 23:02:38.663792 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:38.663619    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy" containerID="cri-o://02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b" gracePeriod=30
Sep 15 23:02:43.913777 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:43.913748    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 23:02:44.002788 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.002750    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istiod-ca-cert\") pod \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") "
Sep 15 23:02:44.002788 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.002790    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-data\") pod \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") "
Sep 15 23:02:44.003066 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.002813    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-podinfo\") pod \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") "
Sep 15 23:02:44.003066 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.002883    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-token\") pod \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") "
Sep 15 23:02:44.003066 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.002919    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wlng\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-kube-api-access-4wlng\") pod \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") "
Sep 15 23:02:44.003066 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.002946    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-credential-socket\") pod \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") "
Sep 15 23:02:44.003066 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.002985    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-certs\") pod \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") "
Sep 15 23:02:44.003066 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003019    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-socket\") pod \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") "
Sep 15 23:02:44.003066 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003043    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-envoy\") pod \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\" (UID: \"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2\") "
Sep 15 23:02:44.003467 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003141    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" (UID: "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:44.003467 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003296    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" (UID: "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:44.003467 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003325    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" (UID: "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:02:44.003467 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003371    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" (UID: "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:44.003467 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003394    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" (UID: "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:44.003979 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003886    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-credential-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:44.003979 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003914    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:44.003979 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003928    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-workload-socket\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:44.003979 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003941    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istiod-ca-cert\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:44.003979 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.003956    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-data\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:44.005574 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.005541    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-kube-api-access-4wlng" (OuterVolumeSpecName: "kube-api-access-4wlng") pod "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" (UID: "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2"). InnerVolumeSpecName "kube-api-access-4wlng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:02:44.005758 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.005740    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" (UID: "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 23:02:44.005848 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.005829    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" (UID: "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:02:44.005886 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.005856    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" (UID: "6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:02:44.038732 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.038700    2578 generic.go:356] "Generic (PLEG): container finished" podID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerID="02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b" exitCode=0
Sep 15 23:02:44.038924 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.038750    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"
Sep 15 23:02:44.038924 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.038783    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" event={"ID":"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2","Type":"ContainerDied","Data":"02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b"}
Sep 15 23:02:44.038924 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.038848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw" event={"ID":"6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2","Type":"ContainerDied","Data":"ea599e713bc7055f786124b66a678b5da83493f5e730f31345517de1c8ae479d"}
Sep 15 23:02:44.038924 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.038878    2578 scope.go:117] "RemoveContainer" containerID="02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b"
Sep 15 23:02:44.048532 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.048514    2578 scope.go:117] "RemoveContainer" containerID="c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634"
Sep 15 23:02:44.056782 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.056762    2578 scope.go:117] "RemoveContainer" containerID="02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b"
Sep 15 23:02:44.057072 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:02:44.057052    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b\": container with ID starting with 02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b not found: ID does not exist" containerID="02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b"
Sep 15 23:02:44.057249 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.057081    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b"} err="failed to get container status \"02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b\": rpc error: code = NotFound desc = could not find container \"02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b\": container with ID starting with 02989438f2e6f67c827012f894715caa7bed213d51d51110e1de1dee8d63301b not found: ID does not exist"
Sep 15 23:02:44.057249 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.057120    2578 scope.go:117] "RemoveContainer" containerID="c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634"
Sep 15 23:02:44.057375 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:02:44.057357    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634\": container with ID starting with c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634 not found: ID does not exist" containerID="c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634"
Sep 15 23:02:44.057413 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.057381    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634"} err="failed to get container status \"c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634\": rpc error: code = NotFound desc = could not find container \"c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634\": container with ID starting with c954080e38d767171dd61e5932becba9c2904a0bc96ae99ce50dfed38a12e634 not found: ID does not exist"
Sep 15 23:02:44.061943 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.061915    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"]
Sep 15 23:02:44.065515 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.065492    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-92482b-openshift-default-54dd8b75b8-kzbzw"]
Sep 15 23:02:44.105514 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.105465    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-token\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:44.105514 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.105504    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wlng\" (UniqueName: \"kubernetes.io/projected/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-kube-api-access-4wlng\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:44.105514 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.105519    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-envoy\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:44.105776 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.105535    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2-istio-podinfo\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:02:44.762175 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:02:44.762140    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" path="/var/lib/kubelet/pods/6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2/volumes"
Sep 15 23:06:41.031750 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:06:41.031672    2578 scope.go:117] "RemoveContainer" containerID="ccfcf13f431dd29e944fba3e5fa811acfafcd94724f4759891e5a1dd36e62cae"
Sep 15 23:06:41.039659 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:06:41.039637    2578 scope.go:117] "RemoveContainer" containerID="8f16ae0142b6c260ea1fa1eef3a338af7d04418053ef65ffcb25309ab07854fd"
Sep 15 23:07:43.270160 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270123    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"]
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270477    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270498    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270509    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dadb847-ce85-4126-a074-93af34fbbedd" containerName="main"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270514    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dadb847-ce85-4126-a074-93af34fbbedd" containerName="main"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270521    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a635b117-00e1-4357-a8ef-e2a0caf593d8" containerName="maas-api"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270528    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a635b117-00e1-4357-a8ef-e2a0caf593d8" containerName="maas-api"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270538    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270547    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270556    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0991a553-ad38-4115-bb53-8ebf7efac73e" containerName="cleanup"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270561    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0991a553-ad38-4115-bb53-8ebf7efac73e" containerName="cleanup"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270569    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06bdaa60-a5cc-4292-8be2-3bd923e5d146" containerName="maas-api"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270577    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06bdaa60-a5cc-4292-8be2-3bd923e5d146" containerName="maas-api"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270592    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270597    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270610    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1c6aa9-8f31-4758-bc34-c50c217a13f4" containerName="cleanup"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270618    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1c6aa9-8f31-4758-bc34-c50c217a13f4" containerName="cleanup"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270633    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270640    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270656    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270665    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270725    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270737    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270743    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270749    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270754    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a635b117-00e1-4357-a8ef-e2a0caf593d8" containerName="maas-api"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270759    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dadb847-ce85-4126-a074-93af34fbbedd" containerName="main"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270764    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0991a553-ad38-4115-bb53-8ebf7efac73e" containerName="cleanup"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270769    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1c6aa9-8f31-4758-bc34-c50c217a13f4" containerName="cleanup"
Sep 15 23:07:43.270839 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.270775    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06bdaa60-a5cc-4292-8be2-3bd923e5d146" containerName="maas-api"
Sep 15 23:07:43.275940 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.275902    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.280615 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.280591    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 23:07:43.280796 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.280601    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-582dz\""
Sep 15 23:07:43.280796 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.280601    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-6f420dbe-kserve-self-signed-certs\""
Sep 15 23:07:43.281302 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.281279    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 23:07:43.283076 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.283053    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"]
Sep 15 23:07:43.328553 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.328514    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tls-certs\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.328715 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.328565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-dshm\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.328715 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.328659    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tmp-dir\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.328715 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.328685    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-home\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.328715 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.328706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-model-cache\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.328857 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.328808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwgkq\" (UniqueName: \"kubernetes.io/projected/6965e4d5-5108-4cd8-a926-04484e9e1e3f-kube-api-access-hwgkq\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.429395 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.429354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwgkq\" (UniqueName: \"kubernetes.io/projected/6965e4d5-5108-4cd8-a926-04484e9e1e3f-kube-api-access-hwgkq\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.429584 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.429412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tls-certs\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.429584 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.429435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-dshm\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.429584 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.429476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tmp-dir\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.429584 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.429496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-home\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.429584 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.429524    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-model-cache\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.429929 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.429906    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-model-cache\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.429987 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.429950    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tmp-dir\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.429987 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.429967    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-home\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.431870 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.431848    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-dshm\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.432273 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.432252    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tls-certs\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.437459 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.437440    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwgkq\" (UniqueName: \"kubernetes.io/projected/6965e4d5-5108-4cd8-a926-04484e9e1e3f-kube-api-access-hwgkq\") pod \"e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") " pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.588198 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.588072    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:43.859250 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:43.859220    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"]
Sep 15 23:07:43.860773 ip-10-0-128-247 kubenswrapper[2578]: W0915 23:07:43.860742    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6965e4d5_5108_4cd8_a926_04484e9e1e3f.slice/crio-f56be45b9deb75a2792bdafc0eeb5a45d04dc72c191ab42e040c7f17df4cd3c0 WatchSource:0}: Error finding container f56be45b9deb75a2792bdafc0eeb5a45d04dc72c191ab42e040c7f17df4cd3c0: Status 404 returned error can't find the container with id f56be45b9deb75a2792bdafc0eeb5a45d04dc72c191ab42e040c7f17df4cd3c0
Sep 15 23:07:44.370570 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:44.370535    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx" event={"ID":"6965e4d5-5108-4cd8-a926-04484e9e1e3f","Type":"ContainerStarted","Data":"098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c"}
Sep 15 23:07:44.370570 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:44.370574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx" event={"ID":"6965e4d5-5108-4cd8-a926-04484e9e1e3f","Type":"ContainerStarted","Data":"f56be45b9deb75a2792bdafc0eeb5a45d04dc72c191ab42e040c7f17df4cd3c0"}
Sep 15 23:07:44.387971 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:44.387914    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx" podStartSLOduration=1.387896649 podStartE2EDuration="1.387896649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 23:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 23:07:44.386315583 +0000 UTC m=+2404.139939714" watchObservedRunningTime="2026-09-15 23:07:44.387896649 +0000 UTC m=+2404.141520775"
Sep 15 23:07:53.589159 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:53.589118    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:53.589549 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:53.589175    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:53.601879 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:53.601842    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:07:54.426721 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:07:54.426686    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:08:08.242385 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:08.242292    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"]
Sep 15 23:08:23.261843 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.261803    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx" podUID="6965e4d5-5108-4cd8-a926-04484e9e1e3f" containerName="main" containerID="cri-o://098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c" gracePeriod=45
Sep 15 23:08:23.504554 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.504517    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:08:23.541242 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.541141    2578 generic.go:356] "Generic (PLEG): container finished" podID="6965e4d5-5108-4cd8-a926-04484e9e1e3f" containerID="098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c" exitCode=0
Sep 15 23:08:23.541242 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.541180    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"
Sep 15 23:08:23.541242 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.541224    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx" event={"ID":"6965e4d5-5108-4cd8-a926-04484e9e1e3f","Type":"ContainerDied","Data":"098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c"}
Sep 15 23:08:23.541474 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.541270    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx" event={"ID":"6965e4d5-5108-4cd8-a926-04484e9e1e3f","Type":"ContainerDied","Data":"f56be45b9deb75a2792bdafc0eeb5a45d04dc72c191ab42e040c7f17df4cd3c0"}
Sep 15 23:08:23.541474 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.541288    2578 scope.go:117] "RemoveContainer" containerID="098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c"
Sep 15 23:08:23.550624 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.550603    2578 scope.go:117] "RemoveContainer" containerID="098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c"
Sep 15 23:08:23.550937 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:08:23.550918    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c\": container with ID starting with 098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c not found: ID does not exist" containerID="098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c"
Sep 15 23:08:23.551011 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.550945    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c"} err="failed to get container status \"098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c\": rpc error: code = NotFound desc = could not find container \"098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c\": container with ID starting with 098d385cc9bdcc8238cabba2a32f48b2bd1938cd61e466679958e78e922eae8c not found: ID does not exist"
Sep 15 23:08:23.585227 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585192    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-model-cache\") pod \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") "
Sep 15 23:08:23.585411 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585249    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-home\") pod \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") "
Sep 15 23:08:23.585411 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585286    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tls-certs\") pod \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") "
Sep 15 23:08:23.585411 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585323    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-dshm\") pod \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") "
Sep 15 23:08:23.585411 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585372    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tmp-dir\") pod \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") "
Sep 15 23:08:23.585411 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585402    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwgkq\" (UniqueName: \"kubernetes.io/projected/6965e4d5-5108-4cd8-a926-04484e9e1e3f-kube-api-access-hwgkq\") pod \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\" (UID: \"6965e4d5-5108-4cd8-a926-04484e9e1e3f\") "
Sep 15 23:08:23.585660 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585512    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-home" (OuterVolumeSpecName: "home") pod "6965e4d5-5108-4cd8-a926-04484e9e1e3f" (UID: "6965e4d5-5108-4cd8-a926-04484e9e1e3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:08:23.585660 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585521    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6965e4d5-5108-4cd8-a926-04484e9e1e3f" (UID: "6965e4d5-5108-4cd8-a926-04484e9e1e3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:08:23.585766 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585663    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6965e4d5-5108-4cd8-a926-04484e9e1e3f" (UID: "6965e4d5-5108-4cd8-a926-04484e9e1e3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:08:23.585806 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585789    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-model-cache\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:23.585841 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585808    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-home\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:23.585875 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.585856    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tmp-dir\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:23.587624 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.587595    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6965e4d5-5108-4cd8-a926-04484e9e1e3f-kube-api-access-hwgkq" (OuterVolumeSpecName: "kube-api-access-hwgkq") pod "6965e4d5-5108-4cd8-a926-04484e9e1e3f" (UID: "6965e4d5-5108-4cd8-a926-04484e9e1e3f"). InnerVolumeSpecName "kube-api-access-hwgkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:08:23.587624 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.587612    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-dshm" (OuterVolumeSpecName: "dshm") pod "6965e4d5-5108-4cd8-a926-04484e9e1e3f" (UID: "6965e4d5-5108-4cd8-a926-04484e9e1e3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:08:23.587757 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.587659    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6965e4d5-5108-4cd8-a926-04484e9e1e3f" (UID: "6965e4d5-5108-4cd8-a926-04484e9e1e3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:08:23.686428 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.686391    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwgkq\" (UniqueName: \"kubernetes.io/projected/6965e4d5-5108-4cd8-a926-04484e9e1e3f-kube-api-access-hwgkq\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:23.686428 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.686423    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6965e4d5-5108-4cd8-a926-04484e9e1e3f-tls-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:23.686428 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.686433    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6965e4d5-5108-4cd8-a926-04484e9e1e3f-dshm\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:23.863154 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.863118    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"]
Sep 15 23:08:23.869176 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:23.869142    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-6f420dbe-kserve-5f5cf866f9-mcvlx"]
Sep 15 23:08:24.762859 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:24.762826    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6965e4d5-5108-4cd8-a926-04484e9e1e3f" path="/var/lib/kubelet/pods/6965e4d5-5108-4cd8-a926-04484e9e1e3f/volumes"
Sep 15 23:08:31.344559 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.344515    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"]
Sep 15 23:08:31.344976 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.344874    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy"
Sep 15 23:08:31.344976 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.344889    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e12005-9215-41e3-8808-c54d4bfca288" containerName="istio-proxy"
Sep 15 23:08:31.344976 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.344915    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6965e4d5-5108-4cd8-a926-04484e9e1e3f" containerName="main"
Sep 15 23:08:31.344976 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.344920    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6965e4d5-5108-4cd8-a926-04484e9e1e3f" containerName="main"
Sep 15 23:08:31.344976 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.344969    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b838f018-d61f-49e5-9fdb-991bcb74bbe7" containerName="istio-proxy"
Sep 15 23:08:31.344976 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.344978    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be7eeef-fc8e-4b9f-b1c1-aa045d2872c2" containerName="istio-proxy"
Sep 15 23:08:31.345201 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.344987    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6965e4d5-5108-4cd8-a926-04484e9e1e3f" containerName="main"
Sep 15 23:08:31.349768 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.349727    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.354329 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.354154    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 23:08:31.354329 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.354184    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tsrzq\""
Sep 15 23:08:31.354329 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.354217    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 15 23:08:31.354329 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.354290    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 23:08:31.354329 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.354298    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 23:08:31.359958 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.359933    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"]
Sep 15 23:08:31.363140 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.363112    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"]
Sep 15 23:08:31.366595 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.366572    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.378637 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.378540    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"]
Sep 15 23:08:31.456511 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.456469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-openshift-service-ca\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.456701 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.456518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-maas-api-tls\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.456701 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.456636    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59lmg\" (UniqueName: \"kubernetes.io/projected/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-kube-api-access-59lmg\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.456701 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.456681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcx9h\" (UniqueName: \"kubernetes.io/projected/57eb6142-0c7a-45f8-b898-d42eebaec2d7-kube-api-access-zcx9h\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.456830 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.456713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57eb6142-0c7a-45f8-b898-d42eebaec2d7-openshift-service-ca\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.456830 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.456736    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-metrics-certs\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.456830 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.456788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-metrics-certs\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.456830 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.456809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-maas-api-tls\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.557243 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.557205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-openshift-service-ca\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.557243 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.557244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-maas-api-tls\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.557475 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.557287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59lmg\" (UniqueName: \"kubernetes.io/projected/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-kube-api-access-59lmg\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.557475 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.557310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcx9h\" (UniqueName: \"kubernetes.io/projected/57eb6142-0c7a-45f8-b898-d42eebaec2d7-kube-api-access-zcx9h\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.557475 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.557338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57eb6142-0c7a-45f8-b898-d42eebaec2d7-openshift-service-ca\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.557475 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.557358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-metrics-certs\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.557475 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.557396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-metrics-certs\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.557475 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.557427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-maas-api-tls\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.558038 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.558013    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-openshift-service-ca\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.558179 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.558068    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57eb6142-0c7a-45f8-b898-d42eebaec2d7-openshift-service-ca\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.559944 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.559920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-maas-api-tls\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.560044 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.559920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-metrics-certs\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.560282 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.560263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-metrics-certs\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.560347 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.560333    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-maas-api-tls\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.565985 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.565953    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcx9h\" (UniqueName: \"kubernetes.io/projected/57eb6142-0c7a-45f8-b898-d42eebaec2d7-kube-api-access-zcx9h\") pod \"maas-api-98857cbf4-l67ll\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.566534 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.566508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59lmg\" (UniqueName: \"kubernetes.io/projected/36b792fb-5d2b-4cb5-b1f7-675c9e21833d-kube-api-access-59lmg\") pod \"maas-api-79d555d56b-s9qb7\" (UID: \"36b792fb-5d2b-4cb5-b1f7-675c9e21833d\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.663295 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.663194    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:31.679638 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.679606    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:31.938495 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.938464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"]
Sep 15 23:08:31.940439 ip-10-0-128-247 kubenswrapper[2578]: W0915 23:08:31.940416    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57eb6142_0c7a_45f8_b898_d42eebaec2d7.slice/crio-18a6580cb1d3ed711985e2dc46e9179f513d1a858c9b8ee2b6ddb82f954de43e WatchSource:0}: Error finding container 18a6580cb1d3ed711985e2dc46e9179f513d1a858c9b8ee2b6ddb82f954de43e: Status 404 returned error can't find the container with id 18a6580cb1d3ed711985e2dc46e9179f513d1a858c9b8ee2b6ddb82f954de43e
Sep 15 23:08:31.941946 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.941919    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 23:08:31.963312 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:31.963281    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"]
Sep 15 23:08:31.965477 ip-10-0-128-247 kubenswrapper[2578]: W0915 23:08:31.965455    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b792fb_5d2b_4cb5_b1f7_675c9e21833d.slice/crio-6864a99ebfd02ecb866914a0154c1b3da4431d7ae031455018a8e93738762230 WatchSource:0}: Error finding container 6864a99ebfd02ecb866914a0154c1b3da4431d7ae031455018a8e93738762230: Status 404 returned error can't find the container with id 6864a99ebfd02ecb866914a0154c1b3da4431d7ae031455018a8e93738762230
Sep 15 23:08:32.584649 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:32.584601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll" event={"ID":"57eb6142-0c7a-45f8-b898-d42eebaec2d7","Type":"ContainerStarted","Data":"71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5"}
Sep 15 23:08:32.585058 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:32.584702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll" event={"ID":"57eb6142-0c7a-45f8-b898-d42eebaec2d7","Type":"ContainerStarted","Data":"18a6580cb1d3ed711985e2dc46e9179f513d1a858c9b8ee2b6ddb82f954de43e"}
Sep 15 23:08:32.585058 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:32.584736    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:32.586263 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:32.586240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7" event={"ID":"36b792fb-5d2b-4cb5-b1f7-675c9e21833d","Type":"ContainerStarted","Data":"94f0ea0ddeb20a567ac93941a09ac38b0eaa41bdb4e084e6a8a6bc2ca46f00e6"}
Sep 15 23:08:32.586331 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:32.586281    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7" event={"ID":"36b792fb-5d2b-4cb5-b1f7-675c9e21833d","Type":"ContainerStarted","Data":"6864a99ebfd02ecb866914a0154c1b3da4431d7ae031455018a8e93738762230"}
Sep 15 23:08:32.586331 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:32.586301    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:32.603501 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:32.603444    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll" podStartSLOduration=1.149601832 podStartE2EDuration="1.603426326s" totalImagesPullingTime="453.824494ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 23:08:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 23:08:32.600932697 +0000 UTC m=+2452.354556823" watchObservedRunningTime="2026-09-15 23:08:32.603426326 +0000 UTC m=+2452.357050453"
Sep 15 23:08:32.618967 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:32.618913    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7" podStartSLOduration=1.110793203 podStartE2EDuration="1.618894965s" totalImagesPullingTime="508.101762ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 23:08:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 23:08:32.61722459 +0000 UTC m=+2452.370848717" watchObservedRunningTime="2026-09-15 23:08:32.618894965 +0000 UTC m=+2452.372519115"
Sep 15 23:08:38.595682 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:38.595653    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:38.596174 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:38.596133    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79d555d56b-s9qb7"
Sep 15 23:08:38.662735 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:38.662703    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"]
Sep 15 23:08:38.662964 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:38.662942    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll" podUID="57eb6142-0c7a-45f8-b898-d42eebaec2d7" containerName="maas-api" containerID="cri-o://71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5" gracePeriod=30
Sep 15 23:08:38.908270 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:38.908238    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:39.025351 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.025309    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57eb6142-0c7a-45f8-b898-d42eebaec2d7-openshift-service-ca\") pod \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") "
Sep 15 23:08:39.025556 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.025363    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcx9h\" (UniqueName: \"kubernetes.io/projected/57eb6142-0c7a-45f8-b898-d42eebaec2d7-kube-api-access-zcx9h\") pod \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") "
Sep 15 23:08:39.025556 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.025388    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-maas-api-tls\") pod \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") "
Sep 15 23:08:39.025556 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.025446    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-metrics-certs\") pod \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\" (UID: \"57eb6142-0c7a-45f8-b898-d42eebaec2d7\") "
Sep 15 23:08:39.025796 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.025770    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57eb6142-0c7a-45f8-b898-d42eebaec2d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57eb6142-0c7a-45f8-b898-d42eebaec2d7" (UID: "57eb6142-0c7a-45f8-b898-d42eebaec2d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:08:39.027576 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.027546    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57eb6142-0c7a-45f8-b898-d42eebaec2d7" (UID: "57eb6142-0c7a-45f8-b898-d42eebaec2d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:08:39.027683 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.027576    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "57eb6142-0c7a-45f8-b898-d42eebaec2d7" (UID: "57eb6142-0c7a-45f8-b898-d42eebaec2d7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:08:39.027683 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.027659    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57eb6142-0c7a-45f8-b898-d42eebaec2d7-kube-api-access-zcx9h" (OuterVolumeSpecName: "kube-api-access-zcx9h") pod "57eb6142-0c7a-45f8-b898-d42eebaec2d7" (UID: "57eb6142-0c7a-45f8-b898-d42eebaec2d7"). InnerVolumeSpecName "kube-api-access-zcx9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:08:39.126428 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.126388    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57eb6142-0c7a-45f8-b898-d42eebaec2d7-openshift-service-ca\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:39.126428 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.126418    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcx9h\" (UniqueName: \"kubernetes.io/projected/57eb6142-0c7a-45f8-b898-d42eebaec2d7-kube-api-access-zcx9h\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:39.126428 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.126428    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-maas-api-tls\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:39.126428 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.126438    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57eb6142-0c7a-45f8-b898-d42eebaec2d7-metrics-certs\") on node \"ip-10-0-128-247.ec2.internal\" DevicePath \"\""
Sep 15 23:08:39.619006 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.618971    2578 generic.go:356] "Generic (PLEG): container finished" podID="57eb6142-0c7a-45f8-b898-d42eebaec2d7" containerID="71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5" exitCode=0
Sep 15 23:08:39.619474 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.619017    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"
Sep 15 23:08:39.619474 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.619056    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll" event={"ID":"57eb6142-0c7a-45f8-b898-d42eebaec2d7","Type":"ContainerDied","Data":"71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5"}
Sep 15 23:08:39.619474 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.619136    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-98857cbf4-l67ll" event={"ID":"57eb6142-0c7a-45f8-b898-d42eebaec2d7","Type":"ContainerDied","Data":"18a6580cb1d3ed711985e2dc46e9179f513d1a858c9b8ee2b6ddb82f954de43e"}
Sep 15 23:08:39.619474 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.619164    2578 scope.go:117] "RemoveContainer" containerID="71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5"
Sep 15 23:08:39.628263 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.628244    2578 scope.go:117] "RemoveContainer" containerID="71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5"
Sep 15 23:08:39.628523 ip-10-0-128-247 kubenswrapper[2578]: E0915 23:08:39.628504    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5\": container with ID starting with 71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5 not found: ID does not exist" containerID="71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5"
Sep 15 23:08:39.628588 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.628531    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5"} err="failed to get container status \"71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5\": rpc error: code = NotFound desc = could not find container \"71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5\": container with ID starting with 71d0e01c6fe4a3e82f556f0a3db720b6d6f874e6d5fb396bcf186ba2f1ae7cd5 not found: ID does not exist"
Sep 15 23:08:39.640270 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.640242    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"]
Sep 15 23:08:39.644657 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:39.644629    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-l67ll"]
Sep 15 23:08:40.763290 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:08:40.763258    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57eb6142-0c7a-45f8-b898-d42eebaec2d7" path="/var/lib/kubelet/pods/57eb6142-0c7a-45f8-b898-d42eebaec2d7/volumes"
Sep 15 23:09:07.702609 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:07.702576    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lvld8_f1961fc4-d87b-40ec-86b6-399d13c0b2ef/server/0.log"
Sep 15 23:09:07.823150 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:07.823118    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xpngn_f1934af5-9eec-4282-8ab4-b814bf55accb/manager/0.log"
Sep 15 23:09:08.157984 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:08.157907    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749d6986f8-rl9lz_45abfd8e-876c-49f2-965e-fc83d7547be3/copy-manifests/0.log"
Sep 15 23:09:08.164666 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:08.164626    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749d6986f8-rl9lz_45abfd8e-876c-49f2-965e-fc83d7547be3/manager/0.log"
Sep 15 23:09:08.619436 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:08.619406    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-79d555d56b-s9qb7_36b792fb-5d2b-4cb5-b1f7-675c9e21833d/maas-api/0.log"
Sep 15 23:09:08.725186 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:08.725156    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29825205-s5l2g_1f480852-191c-4f26-af93-622acfa750fe/cleanup/0.log"
Sep 15 23:09:10.774657 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:10.774627    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8_82dda82c-dff6-4b63-ba02-e4101d73d9ad/istio-proxy/0.log"
Sep 15 23:09:10.888955 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:10.888921    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fflrv_f8943a17-8da5-42be-aeee-4f8da22425d4/discovery/0.log"
Sep 15 23:09:10.998482 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:10.998455    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f98985fb-kt9sm_c7f49c5f-9f71-4029-9e3e-f3977829ecf8/kube-auth-proxy/0.log"
Sep 15 23:09:20.344764 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:20.344700    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dssx6_0b32c671-b247-4256-878e-a286c276b9ca/global-pull-secret-syncer/0.log"
Sep 15 23:09:20.491721 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:20.491690    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m4l8b_0d8ec9c1-59c0-459e-b19b-b86f00b0b55f/konnectivity-agent/0.log"
Sep 15 23:09:20.530296 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:20.530264    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-247.ec2.internal_146a57ab6e613bbac55bc1f7a3764a51/haproxy/0.log"
Sep 15 23:09:27.273145 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.273116    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7cc447d874-cknv6_03fd2ff5-486f-495c-b00a-dc31cc78c6f3/metrics-server/0.log"
Sep 15 23:09:27.297036 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.296964    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-s595m_2dca0af8-0bb4-4c78-b365-5a2be92f4c5c/monitoring-plugin/0.log"
Sep 15 23:09:27.324990 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.324954    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8vldl_1686a97a-f7a6-4f90-8afa-082636493d94/node-exporter/0.log"
Sep 15 23:09:27.344368 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.344336    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8vldl_1686a97a-f7a6-4f90-8afa-082636493d94/kube-rbac-proxy/0.log"
Sep 15 23:09:27.367770 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.367741    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8vldl_1686a97a-f7a6-4f90-8afa-082636493d94/init-textfile/0.log"
Sep 15 23:09:27.639726 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.639648    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fecd4a63-3a50-4e24-88c0-9b278b5e8d30/prometheus/0.log"
Sep 15 23:09:27.660822 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.660794    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fecd4a63-3a50-4e24-88c0-9b278b5e8d30/config-reloader/0.log"
Sep 15 23:09:27.682367 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.682337    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fecd4a63-3a50-4e24-88c0-9b278b5e8d30/thanos-sidecar/0.log"
Sep 15 23:09:27.701919 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.701883    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fecd4a63-3a50-4e24-88c0-9b278b5e8d30/kube-rbac-proxy-web/0.log"
Sep 15 23:09:27.724677 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.724652    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fecd4a63-3a50-4e24-88c0-9b278b5e8d30/kube-rbac-proxy/0.log"
Sep 15 23:09:27.745686 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.745655    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fecd4a63-3a50-4e24-88c0-9b278b5e8d30/kube-rbac-proxy-thanos/0.log"
Sep 15 23:09:27.765446 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.765415    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fecd4a63-3a50-4e24-88c0-9b278b5e8d30/init-config-reloader/0.log"
Sep 15 23:09:27.803234 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.803204    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hn2mz_99cff14d-1671-4944-aa51-51baab7a2d81/prometheus-operator/0.log"
Sep 15 23:09:27.825340 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:27.825317    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hn2mz_99cff14d-1671-4944-aa51-51baab7a2d81/kube-rbac-proxy/0.log"
Sep 15 23:09:28.020675 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:28.020647    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8cd6b5b54-rspv2_287cae11-8551-4505-aebe-9f1623157a9a/thanos-query/0.log"
Sep 15 23:09:28.041235 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:28.041208    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8cd6b5b54-rspv2_287cae11-8551-4505-aebe-9f1623157a9a/kube-rbac-proxy-web/0.log"
Sep 15 23:09:28.061114 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:28.061064    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8cd6b5b54-rspv2_287cae11-8551-4505-aebe-9f1623157a9a/kube-rbac-proxy/0.log"
Sep 15 23:09:28.082062 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:28.082032    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8cd6b5b54-rspv2_287cae11-8551-4505-aebe-9f1623157a9a/prom-label-proxy/0.log"
Sep 15 23:09:28.103027 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:28.102999    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8cd6b5b54-rspv2_287cae11-8551-4505-aebe-9f1623157a9a/kube-rbac-proxy-rules/0.log"
Sep 15 23:09:28.125923 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:28.125897    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8cd6b5b54-rspv2_287cae11-8551-4505-aebe-9f1623157a9a/kube-rbac-proxy-metrics/0.log"
Sep 15 23:09:30.366543 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.366505    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"]
Sep 15 23:09:30.366950 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.366889    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57eb6142-0c7a-45f8-b898-d42eebaec2d7" containerName="maas-api"
Sep 15 23:09:30.366950 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.366901    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="57eb6142-0c7a-45f8-b898-d42eebaec2d7" containerName="maas-api"
Sep 15 23:09:30.367031 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.366954    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="57eb6142-0c7a-45f8-b898-d42eebaec2d7" containerName="maas-api"
Sep 15 23:09:30.369861 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.369844    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.372422 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.372399    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ctgsf\"/\"kube-root-ca.crt\""
Sep 15 23:09:30.372591 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.372406    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ctgsf\"/\"default-dockercfg-drr56\""
Sep 15 23:09:30.373612 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.373596    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ctgsf\"/\"openshift-service-ca.crt\""
Sep 15 23:09:30.381720 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.381692    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"]
Sep 15 23:09:30.476241 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.476207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md8z7\" (UniqueName: \"kubernetes.io/projected/bc962caa-1601-4447-a794-8fdad626befa-kube-api-access-md8z7\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.476241 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.476249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-lib-modules\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.476492 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.476276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-podres\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.476492 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.476373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-sys\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.476492 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.476420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-proc\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.577538 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.577500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-podres\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.577676 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.577571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-sys\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.577676 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.577623    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-proc\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.577795 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.577668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-podres\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.577795 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.577696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md8z7\" (UniqueName: \"kubernetes.io/projected/bc962caa-1601-4447-a794-8fdad626befa-kube-api-access-md8z7\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.577795 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.577698    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-sys\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.577795 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.577732    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-proc\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.577795 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.577734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-lib-modules\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.578028 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.577831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc962caa-1601-4447-a794-8fdad626befa-lib-modules\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.588248 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.588215    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md8z7\" (UniqueName: \"kubernetes.io/projected/bc962caa-1601-4447-a794-8fdad626befa-kube-api-access-md8z7\") pod \"perf-node-gather-daemonset-w568k\" (UID: \"bc962caa-1601-4447-a794-8fdad626befa\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.679515 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.679408    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:30.951332 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:30.951299    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"]
Sep 15 23:09:30.953657 ip-10-0-128-247 kubenswrapper[2578]: W0915 23:09:30.953628    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbc962caa_1601_4447_a794_8fdad626befa.slice/crio-debccc5810a2812d0863c073b614312b23c6c487e898e1972f8bca2eee239597 WatchSource:0}: Error finding container debccc5810a2812d0863c073b614312b23c6c487e898e1972f8bca2eee239597: Status 404 returned error can't find the container with id debccc5810a2812d0863c073b614312b23c6c487e898e1972f8bca2eee239597
Sep 15 23:09:31.857673 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:31.857638    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k" event={"ID":"bc962caa-1601-4447-a794-8fdad626befa","Type":"ContainerStarted","Data":"dd9a2c0188ca4bb4b003bb1e1f080ca4ad4594064704fa20af4b9a9e1d168c9b"}
Sep 15 23:09:31.858083 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:31.857693    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k" event={"ID":"bc962caa-1601-4447-a794-8fdad626befa","Type":"ContainerStarted","Data":"debccc5810a2812d0863c073b614312b23c6c487e898e1972f8bca2eee239597"}
Sep 15 23:09:31.858083 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:31.857855    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:31.876293 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:31.876228    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k" podStartSLOduration=1.8762062149999998 podStartE2EDuration="1.876206215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 23:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 23:09:31.874022125 +0000 UTC m=+2511.627646255" watchObservedRunningTime="2026-09-15 23:09:31.876206215 +0000 UTC m=+2511.629830343"
Sep 15 23:09:32.533880 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:32.533798    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-crlld_1604b4fa-cf86-41f4-9a0d-0db55bad3f65/dns/0.log"
Sep 15 23:09:32.555663 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:32.555629    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-crlld_1604b4fa-cf86-41f4-9a0d-0db55bad3f65/kube-rbac-proxy/0.log"
Sep 15 23:09:32.691464 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:32.691435    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gd8k6_dca0482c-e29e-4e5b-b0e5-0e069d81fdba/dns-node-resolver/0.log"
Sep 15 23:09:33.256278 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:33.256241    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-77pzk_d191e41f-4cad-467f-9bdc-86a9b2d61d19/node-ca/0.log"
Sep 15 23:09:34.236879 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:34.236842    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8_82dda82c-dff6-4b63-ba02-e4101d73d9ad/istio-proxy/0.log"
Sep 15 23:09:34.644600 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:34.644509    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fflrv_f8943a17-8da5-42be-aeee-4f8da22425d4/discovery/0.log"
Sep 15 23:09:34.664720 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:34.664692    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f98985fb-kt9sm_c7f49c5f-9f71-4029-9e3e-f3977829ecf8/kube-auth-proxy/0.log"
Sep 15 23:09:35.658933 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:35.658903    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6qwth_f46c2b32-1ee3-4064-9065-911fdae76f42/serve-healthcheck-canary/0.log"
Sep 15 23:09:36.274301 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:36.274275    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pmjjm_6fc4065e-13df-4e73-906a-55b17f8ab389/kube-rbac-proxy/0.log"
Sep 15 23:09:36.295539 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:36.295512    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pmjjm_6fc4065e-13df-4e73-906a-55b17f8ab389/exporter/0.log"
Sep 15 23:09:36.314111 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:36.314069    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pmjjm_6fc4065e-13df-4e73-906a-55b17f8ab389/extractor/0.log"
Sep 15 23:09:37.872505 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:37.872468    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-w568k"
Sep 15 23:09:38.886298 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:38.886268    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lvld8_f1961fc4-d87b-40ec-86b6-399d13c0b2ef/server/0.log"
Sep 15 23:09:38.968073 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:38.968036    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xpngn_f1934af5-9eec-4282-8ab4-b814bf55accb/manager/0.log"
Sep 15 23:09:39.147488 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:39.147404    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749d6986f8-rl9lz_45abfd8e-876c-49f2-965e-fc83d7547be3/manager/0.log"
Sep 15 23:09:39.169130 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:39.169081    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749d6986f8-rl9lz_45abfd8e-876c-49f2-965e-fc83d7547be3/copy-manifests/0.log"
Sep 15 23:09:40.451735 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:40.451695    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7bd6c5dcdd-hw5jd_0da62b8b-6d6b-48b5-a2e5-458a00b4726d/manager/0.log"
Sep 15 23:09:40.481691 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:40.481659    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8_82dda82c-dff6-4b63-ba02-e4101d73d9ad/istio-proxy/0.log"
Sep 15 23:09:40.499146 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:40.499118    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-n64c7_8cb72563-5068-4515-98cd-395d5593ea19/openshift-lws-operator/0.log"
Sep 15 23:09:40.905795 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:40.905765    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fflrv_f8943a17-8da5-42be-aeee-4f8da22425d4/discovery/0.log"
Sep 15 23:09:40.918227 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:40.918203    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f98985fb-kt9sm_c7f49c5f-9f71-4029-9e3e-f3977829ecf8/kube-auth-proxy/0.log"
Sep 15 23:09:42.512705 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:42.512676    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c5kdp8_82dda82c-dff6-4b63-ba02-e4101d73d9ad/istio-proxy/0.log"
Sep 15 23:09:42.925789 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:42.925703    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fflrv_f8943a17-8da5-42be-aeee-4f8da22425d4/discovery/0.log"
Sep 15 23:09:42.949966 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:42.949935    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f98985fb-kt9sm_c7f49c5f-9f71-4029-9e3e-f3977829ecf8/kube-auth-proxy/0.log"
Sep 15 23:09:47.325643 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.325614    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-slcfb_7c1381d9-2208-4034-89c3-458f4f74eb07/kube-multus-additional-cni-plugins/0.log"
Sep 15 23:09:47.346418 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.346368    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-slcfb_7c1381d9-2208-4034-89c3-458f4f74eb07/egress-router-binary-copy/0.log"
Sep 15 23:09:47.368791 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.368764    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-slcfb_7c1381d9-2208-4034-89c3-458f4f74eb07/cni-plugins/0.log"
Sep 15 23:09:47.389443 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.389400    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-slcfb_7c1381d9-2208-4034-89c3-458f4f74eb07/bond-cni-plugin/0.log"
Sep 15 23:09:47.408985 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.408939    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-slcfb_7c1381d9-2208-4034-89c3-458f4f74eb07/routeoverride-cni/0.log"
Sep 15 23:09:47.428794 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.428767    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-slcfb_7c1381d9-2208-4034-89c3-458f4f74eb07/whereabouts-cni-bincopy/0.log"
Sep 15 23:09:47.447556 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.447471    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-slcfb_7c1381d9-2208-4034-89c3-458f4f74eb07/whereabouts-cni/0.log"
Sep 15 23:09:47.576361 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.576305    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zmr42_002b8e1b-4e2e-44cd-b110-24f8ff8cd2dc/kube-multus/0.log"
Sep 15 23:09:47.678683 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.678640    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tbgzs_7b02d944-b4ba-440f-8ec2-bec251299069/network-metrics-daemon/0.log"
Sep 15 23:09:47.696922 ip-10-0-128-247 kubenswrapper[2578]: I0915 23:09:47.696898    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tbgzs_7b02d944-b4ba-440f-8ec2-bec251299069/kube-rbac-proxy/0.log"